2026-02-23T08:05:23.902 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-23T08:05:23.918 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-23T08:05:23.995 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-23_07:12:59-rados-wip-bharath8-testing-2026-02-22-0716-distro-default-trial/65627 branch: wip-bharath8-testing-2026-02-22-0716 description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '65627' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: skanta-2026-02-23_07:12:59-rados-wip-bharath8-testing-2026-02-22-0716-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath8-testing-2026-02-22-0716 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 5adce3ff0fe1645dcbc38d059f948571dc3d69da ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 5adce3ff0fe1645dcbc38d059f948571dc3d69da workunit: branch: wip-bharath8-testing-2026-02-22-0716 sha1: 5adce3ff0fe1645dcbc38d059f948571dc3d69da owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 5057 sha1: 5adce3ff0fe1645dcbc38d059f948571dc3d69da sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath8-testing-2026-02-22-0716 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_5adce3ff0fe1645dcbc38d059f948571dc3d69da/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 5adce3ff0fe1645dcbc38d059f948571dc3d69da targets: trial186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNt5fNbBA3O5RXqJ72HpJ2ODlcUzB7kocWVz8O9NO5AtJDMZP/JdZNQQRnuH9XS+4chBVB26rLuhWcgJWHIbAyU= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-23_07:12:59 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-23T08:05:23.995 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_5adce3ff0fe1645dcbc38d059f948571dc3d69da/qa; will attempt to use it 2026-02-23T08:05:23.995 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_5adce3ff0fe1645dcbc38d059f948571dc3d69da/qa/tasks 2026-02-23T08:05:23.996 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-23T08:05:24.001 INFO:teuthology.task.internal:Checking packages... 2026-02-23T08:05:24.001 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '5adce3ff0fe1645dcbc38d059f948571dc3d69da' 2026-02-23T08:05:24.001 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-23T08:05:24.001 INFO:teuthology.packaging:ref: None 2026-02-23T08:05:24.001 INFO:teuthology.packaging:tag: None 2026-02-23T08:05:24.001 INFO:teuthology.packaging:branch: wip-bharath8-testing-2026-02-22-0716 2026-02-23T08:05:24.001 INFO:teuthology.packaging:sha1: 5adce3ff0fe1645dcbc38d059f948571dc3d69da 2026-02-23T08:05:24.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath8-testing-2026-02-22-0716 2026-02-23T08:05:24.124 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5433-g5adce3ff-1jammy 2026-02-23T08:05:24.126 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-23T08:05:24.137 INFO:teuthology.task.internal:no buildpackages task found 2026-02-23T08:05:24.137 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-23T08:05:24.152 INFO:teuthology.task.internal:Saving configuration 2026-02-23T08:05:24.160 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-23T08:05:24.172 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-23T08:05:24.251 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial186.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-23_07:12:59-rados-wip-bharath8-testing-2026-02-22-0716-distro-default-trial/65627', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-23 08:01:54.805656', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMx4UVB/MAPkUDLNKa8qR5MX63NoA4a647SLC2S3VAQI'} 2026-02-23T08:05:24.252 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-23T08:05:24.266 INFO:teuthology.task.internal:roles: ubuntu@trial186.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-23T08:05:24.266 INFO:teuthology.run_tasks:Running task console_log... 2026-02-23T08:05:24.351 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x706b9e6007c0>, signals=[15]) 2026-02-23T08:05:24.351 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-23T08:05:24.363 INFO:teuthology.task.internal:Opening connections... 2026-02-23T08:05:24.363 DEBUG:teuthology.task.internal:connecting to ubuntu@trial186.front.sepia.ceph.com 2026-02-23T08:05:24.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-23T08:05:24.429 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-23T08:05:24.441 DEBUG:teuthology.orchestra.run.trial186:> uname -m 2026-02-23T08:05:24.506 INFO:teuthology.orchestra.run.trial186.stdout:x86_64 2026-02-23T08:05:24.507 DEBUG:teuthology.orchestra.run.trial186:> cat /etc/os-release 2026-02-23T08:05:24.550 INFO:teuthology.orchestra.run.trial186.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-23T08:05:24.550 INFO:teuthology.orchestra.run.trial186.stdout:NAME="Ubuntu" 2026-02-23T08:05:24.550 INFO:teuthology.orchestra.run.trial186.stdout:VERSION_ID="22.04" 2026-02-23T08:05:24.550 INFO:teuthology.orchestra.run.trial186.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-23T08:05:24.550 INFO:teuthology.orchestra.run.trial186.stdout:VERSION_CODENAME=jammy 2026-02-23T08:05:24.551 INFO:teuthology.orchestra.run.trial186.stdout:ID=ubuntu 2026-02-23T08:05:24.551 INFO:teuthology.orchestra.run.trial186.stdout:ID_LIKE=debian 2026-02-23T08:05:24.551 INFO:teuthology.orchestra.run.trial186.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-23T08:05:24.551 INFO:teuthology.orchestra.run.trial186.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-23T08:05:24.551 INFO:teuthology.orchestra.run.trial186.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-23T08:05:24.551 INFO:teuthology.orchestra.run.trial186.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-23T08:05:24.551 INFO:teuthology.orchestra.run.trial186.stdout:UBUNTU_CODENAME=jammy 2026-02-23T08:05:24.551 INFO:teuthology.lock.ops:Updating trial186.front.sepia.ceph.com on lock server 2026-02-23T08:05:24.615 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-23T08:05:24.627 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-23T08:05:24.651 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-23T08:05:24.652 DEBUG:teuthology.orchestra.run.trial186:> test '!' -e /home/ubuntu/cephtest 2026-02-23T08:05:24.656 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-23T08:05:24.668 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-23T08:05:24.668 DEBUG:teuthology.orchestra.run.trial186:> test -z $(ls -A /var/lib/ceph) 2026-02-23T08:05:24.708 INFO:teuthology.orchestra.run.trial186.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-23T08:05:24.708 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-23T08:05:24.790 INFO:teuthology.run_tasks:Running task kernel... 2026-02-23T08:05:24.809 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-23T08:05:24.809 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-23T08:05:24.809 DEBUG:teuthology.orchestra.run.trial186:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-23T08:05:24.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T08:05:24.813 DEBUG:teuthology.orchestra.run.trial186:> uname -r 2026-02-23T08:05:24.859 INFO:teuthology.orchestra.run.trial186.stdout:5.15.0-170-generic 2026-02-23T08:05:24.859 INFO:teuthology.task.kernel:Running kernel on trial186: 5.15.0-170-generic 2026-02-23T08:05:24.859 DEBUG:teuthology.orchestra.run.trial186:> sudo apt-get clean 2026-02-23T08:05:24.917 DEBUG:teuthology.orchestra.run.trial186:> sudo apt-get update 2026-02-23T08:05:25.356 INFO:teuthology.orchestra.run.trial186.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-23T08:05:25.430 INFO:teuthology.orchestra.run.trial186.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-23T08:05:25.703 INFO:teuthology.orchestra.run.trial186.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-23T08:05:25.824 INFO:teuthology.orchestra.run.trial186.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-23T08:05:25.938 INFO:teuthology.orchestra.run.trial186.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-23T08:05:26.267 INFO:teuthology.orchestra.run.trial186.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-23T08:05:26.277 INFO:teuthology.orchestra.run.trial186.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-23T08:05:26.277 INFO:teuthology.orchestra.run.trial186.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-23T08:05:26.305 INFO:teuthology.orchestra.run.trial186.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-23T08:05:26.311 INFO:teuthology.orchestra.run.trial186.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-23T08:05:26.312 INFO:teuthology.orchestra.run.trial186.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-23T08:05:26.440 INFO:teuthology.orchestra.run.trial186.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-23T08:05:26.459 INFO:teuthology.orchestra.run.trial186.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-23T08:05:26.530 INFO:teuthology.orchestra.run.trial186.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-23T08:05:26.538 INFO:teuthology.orchestra.run.trial186.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-23T08:05:26.538 INFO:teuthology.orchestra.run.trial186.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-23T08:05:26.566 INFO:teuthology.orchestra.run.trial186.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-23T08:05:26.566 INFO:teuthology.orchestra.run.trial186.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-23T08:05:26.683 INFO:teuthology.orchestra.run.trial186.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-23T08:05:30.287 INFO:teuthology.orchestra.run.trial186.stdout:Fetched 22.6 MB in 3s (8553 kB/s) 2026-02-23T08:05:30.829 INFO:teuthology.orchestra.run.trial186.stdout:Reading package lists... 2026-02-23T08:05:30.842 DEBUG:teuthology.orchestra.run.trial186:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-23T08:05:30.877 INFO:teuthology.orchestra.run.trial186.stdout:Reading package lists... 2026-02-23T08:05:30.990 INFO:teuthology.orchestra.run.trial186.stdout:Building dependency tree... 2026-02-23T08:05:30.991 INFO:teuthology.orchestra.run.trial186.stdout:Reading state information... 2026-02-23T08:05:31.079 INFO:teuthology.orchestra.run.trial186.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-23T08:05:31.079 INFO:teuthology.orchestra.run.trial186.stdout:linux-image-generic set to manually installed. 2026-02-23T08:05:31.079 INFO:teuthology.orchestra.run.trial186.stdout:The following packages were automatically installed and are no longer required: 2026-02-23T08:05:31.079 INFO:teuthology.orchestra.run.trial186.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-23T08:05:31.079 INFO:teuthology.orchestra.run.trial186.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-23T08:05:31.079 INFO:teuthology.orchestra.run.trial186.stdout: sg3-utils-udev 2026-02-23T08:05:31.079 INFO:teuthology.orchestra.run.trial186.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-23T08:05:31.099 INFO:teuthology.orchestra.run.trial186.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-23T08:05:31.101 DEBUG:teuthology.orchestra.run.trial186:> dpkg -s linux-image-generic 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Package: linux-image-generic 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Status: install ok installed 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Priority: optional 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Section: kernel 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Installed-Size: 21 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Maintainer: Ubuntu Kernel Team 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Architecture: amd64 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Source: linux-meta 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Version: 5.15.0.170.159 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Recommends: thermald 2026-02-23T08:05:31.153 INFO:teuthology.orchestra.run.trial186.stdout:Description: Generic Linux kernel image 2026-02-23T08:05:31.154 INFO:teuthology.orchestra.run.trial186.stdout: This package will always depend on the latest generic kernel image 2026-02-23T08:05:31.154 INFO:teuthology.orchestra.run.trial186.stdout: available. 2026-02-23T08:05:31.154 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-23T08:05:31.154 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-23T08:05:31.154 DEBUG:teuthology.orchestra.run.trial186:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-23T08:05:31.203 INFO:teuthology.orchestra.run.trial186.stdout:ttyS1 2026-02-23T08:05:31.221 DEBUG:teuthology.parallel:result is None 2026-02-23T08:05:31.221 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-23T08:05:31.232 INFO:teuthology.task.internal:Creating test directory... 2026-02-23T08:05:31.232 DEBUG:teuthology.orchestra.run.trial186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-23T08:05:31.266 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-23T08:05:31.281 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-23T08:05:31.293 INFO:teuthology.task.internal:Creating archive directory... 2026-02-23T08:05:31.293 DEBUG:teuthology.orchestra.run.trial186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-23T08:05:31.313 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-23T08:05:31.324 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-23T08:05:31.325 DEBUG:teuthology.orchestra.run.trial186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-23T08:05:31.360 INFO:teuthology.orchestra.run.trial186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:05:31.366 INFO:teuthology.orchestra.run.trial186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:05:31.367 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-23T08:05:31.381 INFO:teuthology.task.internal:Configuring sudo... 2026-02-23T08:05:31.381 DEBUG:teuthology.orchestra.run.trial186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-23T08:05:31.415 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-23T08:05:31.428 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-23T08:05:31.428 DEBUG:teuthology.orchestra.run.trial186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-23T08:05:31.458 DEBUG:teuthology.orchestra.run.trial186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T08:05:31.502 DEBUG:teuthology.orchestra.run.trial186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T08:05:31.546 DEBUG:teuthology.orchestra.run.trial186:> set -ex 2026-02-23T08:05:31.546 DEBUG:teuthology.orchestra.run.trial186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-23T08:05:31.596 DEBUG:teuthology.orchestra.run.trial186:> sudo service rsyslog restart 2026-02-23T08:05:31.656 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-23T08:05:31.668 INFO:teuthology.task.internal:Starting timer... 2026-02-23T08:05:31.668 INFO:teuthology.run_tasks:Running task pcp... 2026-02-23T08:05:31.683 INFO:teuthology.run_tasks:Running task selinux... 2026-02-23T08:05:31.696 DEBUG:teuthology.task.selinux:Excluding trial186: OS 'ubuntu' does not support SELinux 2026-02-23T08:05:31.696 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-23T08:05:31.696 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-23T08:05:31.696 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-23T08:05:31.696 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-23T08:05:31.711 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-23T08:05:31.713 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-23T08:05:36.870 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-23T08:05:36.876 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-23T08:05:36.877 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-23T08:07:56.811 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial186.front.sepia.ceph.com')] 2026-02-23T08:07:56.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial186.front.sepia.ceph.com' 2026-02-23T08:07:56.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-23T08:07:56.884 DEBUG:teuthology.orchestra.run.trial186:> true 2026-02-23T08:07:57.132 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial186.front.sepia.ceph.com' 2026-02-23T08:07:57.132 INFO:teuthology.run_tasks:Running task clock... 2026-02-23T08:07:57.146 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-23T08:07:57.146 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-23T08:07:57.146 DEBUG:teuthology.orchestra.run.trial186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-23T08:07:57.195 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-23T08:07:57.195 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Command line: ntpd -gq 2026-02-23T08:07:57.195 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: ---------------------------------------------------- 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: ntp-4 is maintained by Network Time Foundation, 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: corporation. Support and training for ntp-4 are 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: available at https://www.nwtime.org/support 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: ---------------------------------------------------- 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: proto: precision = 0.030 usec (-25) 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: basedate set to 2022-02-04 2026-02-23T08:07:57.196 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: gps base set to 2022-02-06 (week 2196) 2026-02-23T08:07:57.197 INFO:teuthology.orchestra.run.trial186.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-23T08:07:57.197 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-23T08:07:57.198 INFO:teuthology.orchestra.run.trial186.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: restrict ::: KOD does nothing without LIMITED. 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Listen and drop on 0 v6wildcard [::]:123 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Listen normally on 2 lo 127.0.0.1:123 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Listen normally on 3 enp1s0f0 10.20.193.186:123 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Listen normally on 4 lo [::1]:123 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:6382%2]:123 2026-02-23T08:07:57.199 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:07:57 ntpd[9308]: Listening on routing socket on fd #22 for interface updates 2026-02-23T08:08:01.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:01 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:01.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:01 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:01.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:01 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:01.238 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:01 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:01.238 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:01 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:01.238 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:01 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:02.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:02.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:02.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:02.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:02.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:02.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:02.198 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:02.199 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:02.199 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:02.199 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:02.199 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:02.199 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:02.199 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:02.199 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:02.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:02.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:02.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:02.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:02 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:03.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:03 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:03.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:03 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:03.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:03 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:03.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:03 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:03.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:03 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:03.238 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:03 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:04.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:04.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:04.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:04.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:04.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:04.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:04.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:04.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:04.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:04.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:04 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:05.194 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:05 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:05.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:05 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:05.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:05 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:05.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:05 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:05.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:05 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:05.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:05 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:06.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:06.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:06.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:06.196 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:06.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:06.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:06.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:06.233 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:06 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:07.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:07.195 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: can't open /var/log/ntpstats/rawstats.20260223: Permission denied 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: can't open /var/log/ntpstats/peerstats.20260223: Permission denied 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stdout:23 Feb 08:08:07 ntpd[9308]: ntpd: time set -0.038490 s 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stdout:ntpd: time set -0.038490s 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-23T08:08:07.237 INFO:teuthology.orchestra.run.trial186.stderr:23 Feb 08:08:07 ntpd[9308]: can't open /var/log/ntpstats/loopstats.20260223: Permission denied 2026-02-23T08:08:07.307 INFO:teuthology.orchestra.run.trial186.stdout: remote refid st t when poll reach delay offset jitter 2026-02-23T08:08:07.307 INFO:teuthology.orchestra.run.trial186.stdout:============================================================================== 2026-02-23T08:08:07.307 INFO:teuthology.orchestra.run.trial186.stdout: ool-44c5de70.dy .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-23T08:08:07.307 INFO:teuthology.orchestra.run.trial186.stdout: as393746.mci.tr .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-23T08:08:07.307 INFO:teuthology.orchestra.run.trial186.stdout: 172-233-157-223 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-23T08:08:07.307 INFO:teuthology.orchestra.run.trial186.stdout: sensei.ruselabs .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-23T08:08:07.308 INFO:teuthology.run_tasks:Running task install... 2026-02-23T08:08:07.321 DEBUG:teuthology.task.install:project ceph 2026-02-23T08:08:07.321 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5adce3ff0fe1645dcbc38d059f948571dc3d69da'}} 2026-02-23T08:08:07.321 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5adce3ff0fe1645dcbc38d059f948571dc3d69da'} 2026-02-23T08:08:07.322 INFO:teuthology.task.install:Using flavor: default 2026-02-23T08:08:07.329 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-23T08:08:07.329 INFO:teuthology.task.install:extra packages: [] 2026-02-23T08:08:07.330 DEBUG:teuthology.orchestra.run.trial186:> sudo apt-key list | grep Ceph 2026-02-23T08:08:07.404 INFO:teuthology.orchestra.run.trial186.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-23T08:08:07.433 INFO:teuthology.orchestra.run.trial186.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-23T08:08:07.433 INFO:teuthology.orchestra.run.trial186.stdout:uid [ unknown] Ceph.com (release key) 2026-02-23T08:08:07.434 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-23T08:08:07.435 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=5adce3ff0fe1645dcbc38d059f948571dc3d69da 2026-02-23T08:08:07.590 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default/ 2026-02-23T08:08:07.590 INFO:teuthology.task.install.deb:Package version is 20.3.0-5433-g5adce3ff-1jammy 2026-02-23T08:08:07.973 DEBUG:teuthology.orchestra.run.trial186:> set -ex 2026-02-23T08:08:07.973 DEBUG:teuthology.orchestra.run.trial186:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-23T08:08:07.983 DEBUG:teuthology.orchestra.run.trial186:> sudo apt-get update 2026-02-23T08:08:08.351 INFO:teuthology.orchestra.run.trial186.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy InRelease 2026-02-23T08:08:08.420 INFO:teuthology.orchestra.run.trial186.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-23T08:08:08.490 INFO:teuthology.orchestra.run.trial186.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-23T08:08:08.491 INFO:teuthology.orchestra.run.trial186.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-23T08:08:08.560 INFO:teuthology.orchestra.run.trial186.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-23T08:08:08.575 INFO:teuthology.orchestra.run.trial186.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-23T08:08:08.667 INFO:teuthology.orchestra.run.trial186.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-23T08:08:08.758 INFO:teuthology.orchestra.run.trial186.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-23T08:08:08.899 INFO:teuthology.orchestra.run.trial186.stdout:Fetched 26.9 kB in 1s (35.5 kB/s) 2026-02-23T08:08:09.419 INFO:teuthology.orchestra.run.trial186.stdout:Reading package lists... 2026-02-23T08:08:09.432 DEBUG:teuthology.orchestra.run.trial186:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5433-g5adce3ff-1jammy cephadm=20.3.0-5433-g5adce3ff-1jammy ceph-mds=20.3.0-5433-g5adce3ff-1jammy ceph-mgr=20.3.0-5433-g5adce3ff-1jammy ceph-osd=20.3.0-5433-g5adce3ff-1jammy ceph-osd-classic=20.3.0-5433-g5adce3ff-1jammy ceph-common=20.3.0-5433-g5adce3ff-1jammy ceph-fuse=20.3.0-5433-g5adce3ff-1jammy ceph-test=20.3.0-5433-g5adce3ff-1jammy ceph-volume=20.3.0-5433-g5adce3ff-1jammy radosgw=20.3.0-5433-g5adce3ff-1jammy python3-rados=20.3.0-5433-g5adce3ff-1jammy python3-rgw=20.3.0-5433-g5adce3ff-1jammy python3-cephfs=20.3.0-5433-g5adce3ff-1jammy python3-rbd=20.3.0-5433-g5adce3ff-1jammy libcephfs2=20.3.0-5433-g5adce3ff-1jammy libcephfs-dev=20.3.0-5433-g5adce3ff-1jammy librados2=20.3.0-5433-g5adce3ff-1jammy librbd1=20.3.0-5433-g5adce3ff-1jammy rbd-fuse=20.3.0-5433-g5adce3ff-1jammy 2026-02-23T08:08:09.466 INFO:teuthology.orchestra.run.trial186.stdout:Reading package lists... 2026-02-23T08:08:09.568 INFO:teuthology.orchestra.run.trial186.stdout:Building dependency tree... 2026-02-23T08:08:09.579 INFO:teuthology.orchestra.run.trial186.stdout:Reading state information... 2026-02-23T08:08:09.658 INFO:teuthology.orchestra.run.trial186.stdout:The following packages were automatically installed and are no longer required: 2026-02-23T08:08:09.658 INFO:teuthology.orchestra.run.trial186.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-23T08:08:09.658 INFO:teuthology.orchestra.run.trial186.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-23T08:08:09.658 INFO:teuthology.orchestra.run.trial186.stdout: sg3-utils-udev 2026-02-23T08:08:09.658 INFO:teuthology.orchestra.run.trial186.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout:The following additional packages will be installed: 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-23T08:08:09.659 INFO:teuthology.orchestra.run.trial186.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout:Suggested packages: 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-23T08:08:09.660 INFO:teuthology.orchestra.run.trial186.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-23T08:08:09.661 INFO:teuthology.orchestra.run.trial186.stdout:Recommended packages: 2026-02-23T08:08:09.661 INFO:teuthology.orchestra.run.trial186.stdout: btrfs-tools 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout:The following NEW packages will be installed: 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-23T08:08:09.691 INFO:teuthology.orchestra.run.trial186.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-23T08:08:09.692 INFO:teuthology.orchestra.run.trial186.stdout: socat xmlstarlet 2026-02-23T08:08:09.795 INFO:teuthology.orchestra.run.trial186.stdout:0 upgraded, 87 newly installed, 0 to remove and 13 not upgraded. 2026-02-23T08:08:09.796 INFO:teuthology.orchestra.run.trial186.stdout:Need to get 702 MB of archives. 2026-02-23T08:08:09.796 INFO:teuthology.orchestra.run.trial186.stdout:After this operation, 2671 MB of additional disk space will be used. 2026-02-23T08:08:09.796 INFO:teuthology.orchestra.run.trial186.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-23T08:08:09.842 INFO:teuthology.orchestra.run.trial186.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-23T08:08:09.843 INFO:teuthology.orchestra.run.trial186.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-23T08:08:09.852 INFO:teuthology.orchestra.run.trial186.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-23T08:08:09.881 INFO:teuthology.orchestra.run.trial186.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-23T08:08:09.882 INFO:teuthology.orchestra.run.trial186.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-23T08:08:09.887 INFO:teuthology.orchestra.run.trial186.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-23T08:08:09.889 INFO:teuthology.orchestra.run.trial186.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-23T08:08:09.890 INFO:teuthology.orchestra.run.trial186.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-23T08:08:09.890 INFO:teuthology.orchestra.run.trial186.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-23T08:08:09.891 INFO:teuthology.orchestra.run.trial186.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-23T08:08:09.895 INFO:teuthology.orchestra.run.trial186.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-23T08:08:09.896 INFO:teuthology.orchestra.run.trial186.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-23T08:08:09.896 INFO:teuthology.orchestra.run.trial186.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-23T08:08:09.901 INFO:teuthology.orchestra.run.trial186.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-23T08:08:09.902 INFO:teuthology.orchestra.run.trial186.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-23T08:08:09.902 INFO:teuthology.orchestra.run.trial186.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-23T08:08:09.902 INFO:teuthology.orchestra.run.trial186.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-23T08:08:09.902 INFO:teuthology.orchestra.run.trial186.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-23T08:08:09.911 INFO:teuthology.orchestra.run.trial186.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-23T08:08:09.911 INFO:teuthology.orchestra.run.trial186.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-23T08:08:09.913 INFO:teuthology.orchestra.run.trial186.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-23T08:08:09.913 INFO:teuthology.orchestra.run.trial186.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-23T08:08:09.914 INFO:teuthology.orchestra.run.trial186.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-23T08:08:09.920 INFO:teuthology.orchestra.run.trial186.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-23T08:08:09.921 INFO:teuthology.orchestra.run.trial186.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-23T08:08:09.921 INFO:teuthology.orchestra.run.trial186.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-23T08:08:09.935 INFO:teuthology.orchestra.run.trial186.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-23T08:08:09.937 INFO:teuthology.orchestra.run.trial186.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-23T08:08:09.937 INFO:teuthology.orchestra.run.trial186.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-23T08:08:09.946 INFO:teuthology.orchestra.run.trial186.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-23T08:08:09.947 INFO:teuthology.orchestra.run.trial186.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-23T08:08:09.947 INFO:teuthology.orchestra.run.trial186.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-23T08:08:09.947 INFO:teuthology.orchestra.run.trial186.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-23T08:08:09.947 INFO:teuthology.orchestra.run.trial186.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-23T08:08:09.947 INFO:teuthology.orchestra.run.trial186.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-23T08:08:09.949 INFO:teuthology.orchestra.run.trial186.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-23T08:08:09.949 INFO:teuthology.orchestra.run.trial186.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-23T08:08:09.950 INFO:teuthology.orchestra.run.trial186.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-23T08:08:09.958 INFO:teuthology.orchestra.run.trial186.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-23T08:08:09.960 INFO:teuthology.orchestra.run.trial186.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-23T08:08:09.961 INFO:teuthology.orchestra.run.trial186.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-23T08:08:09.963 INFO:teuthology.orchestra.run.trial186.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-23T08:08:09.964 INFO:teuthology.orchestra.run.trial186.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-23T08:08:09.968 INFO:teuthology.orchestra.run.trial186.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-23T08:08:09.968 INFO:teuthology.orchestra.run.trial186.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-23T08:08:09.968 INFO:teuthology.orchestra.run.trial186.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-23T08:08:09.969 INFO:teuthology.orchestra.run.trial186.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-23T08:08:09.972 INFO:teuthology.orchestra.run.trial186.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-23T08:08:09.977 INFO:teuthology.orchestra.run.trial186.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-23T08:08:09.978 INFO:teuthology.orchestra.run.trial186.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-23T08:08:09.979 INFO:teuthology.orchestra.run.trial186.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-23T08:08:09.979 INFO:teuthology.orchestra.run.trial186.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-23T08:08:10.053 INFO:teuthology.orchestra.run.trial186.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5433-g5adce3ff-1jammy [3795 kB] 2026-02-23T08:08:10.774 INFO:teuthology.orchestra.run.trial186.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5433-g5adce3ff-1jammy [2909 kB] 2026-02-23T08:08:10.960 INFO:teuthology.orchestra.run.trial186.stdout:Get:56 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5433-g5adce3ff-1jammy [882 kB] 2026-02-23T08:08:10.976 INFO:teuthology.orchestra.run.trial186.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5433-g5adce3ff-1jammy [359 kB] 2026-02-23T08:08:11.027 INFO:teuthology.orchestra.run.trial186.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5433-g5adce3ff-1jammy [32.8 kB] 2026-02-23T08:08:11.028 INFO:teuthology.orchestra.run.trial186.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5433-g5adce3ff-1jammy [225 kB] 2026-02-23T08:08:11.032 INFO:teuthology.orchestra.run.trial186.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5433-g5adce3ff-1jammy [88.9 kB] 2026-02-23T08:08:11.034 INFO:teuthology.orchestra.run.trial186.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5433-g5adce3ff-1jammy [339 kB] 2026-02-23T08:08:11.040 INFO:teuthology.orchestra.run.trial186.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5433-g5adce3ff-1jammy [9090 kB] 2026-02-23T08:08:11.634 INFO:teuthology.orchestra.run.trial186.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5433-g5adce3ff-1jammy [112 kB] 2026-02-23T08:08:11.635 INFO:teuthology.orchestra.run.trial186.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5433-g5adce3ff-1jammy [261 kB] 2026-02-23T08:08:11.640 INFO:teuthology.orchestra.run.trial186.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5433-g5adce3ff-1jammy [31.5 MB] 2026-02-23T08:08:13.359 INFO:teuthology.orchestra.run.trial186.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5433-g5adce3ff-1jammy [4983 kB] 2026-02-23T08:08:13.598 INFO:teuthology.orchestra.run.trial186.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5433-g5adce3ff-1jammy [294 kB] 2026-02-23T08:08:13.633 INFO:teuthology.orchestra.run.trial186.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5433-g5adce3ff-1jammy [127 kB] 2026-02-23T08:08:13.641 INFO:teuthology.orchestra.run.trial186.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5433-g5adce3ff-1jammy [922 kB] 2026-02-23T08:08:13.660 INFO:teuthology.orchestra.run.trial186.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5433-g5adce3ff-1jammy [7541 kB] 2026-02-23T08:08:14.011 INFO:teuthology.orchestra.run.trial186.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5433-g5adce3ff-1jammy [468 MB] 2026-02-23T08:11:01.524 INFO:teuthology.orchestra.run.trial186.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5433-g5adce3ff-1jammy [180 kB] 2026-02-23T08:11:01.654 INFO:teuthology.orchestra.run.trial186.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5433-g5adce3ff-1jammy [14.0 kB] 2026-02-23T08:11:01.655 INFO:teuthology.orchestra.run.trial186.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5433-g5adce3ff-1jammy [1030 kB] 2026-02-23T08:11:02.295 INFO:teuthology.orchestra.run.trial186.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5433-g5adce3ff-1jammy [2437 kB] 2026-02-23T08:11:03.517 INFO:teuthology.orchestra.run.trial186.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5433-g5adce3ff-1jammy [1072 kB] 2026-02-23T08:11:04.001 INFO:teuthology.orchestra.run.trial186.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5433-g5adce3ff-1jammy [190 kB] 2026-02-23T08:11:04.084 INFO:teuthology.orchestra.run.trial186.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5433-g5adce3ff-1jammy [55.0 MB] 2026-02-23T08:11:20.215 INFO:teuthology.orchestra.run.trial186.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5433-g5adce3ff-1jammy [8626 kB] 2026-02-23T08:11:21.699 INFO:teuthology.orchestra.run.trial186.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5433-g5adce3ff-1jammy [14.2 kB] 2026-02-23T08:11:21.706 INFO:teuthology.orchestra.run.trial186.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5433-g5adce3ff-1jammy [68.6 MB] 2026-02-23T08:11:36.127 INFO:teuthology.orchestra.run.trial186.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5433-g5adce3ff-1jammy [136 kB] 2026-02-23T08:11:36.128 INFO:teuthology.orchestra.run.trial186.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5433-g5adce3ff-1jammy [47.3 kB] 2026-02-23T08:11:36.161 INFO:teuthology.orchestra.run.trial186.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5433-g5adce3ff-1jammy [33.1 kB] 2026-02-23T08:11:36.161 INFO:teuthology.orchestra.run.trial186.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5433-g5adce3ff-1jammy [44.2 kB] 2026-02-23T08:11:36.162 INFO:teuthology.orchestra.run.trial186.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5433-g5adce3ff-1jammy [18.3 MB] 2026-02-23T08:11:39.430 INFO:teuthology.orchestra.run.trial186.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-bharath8-testing-2026-02-22-0716/5adce3ff0fe1645dcbc38d059f948571dc3d69da/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5433-g5adce3ff-1jammy [93.7 kB] 2026-02-23T08:11:39.713 INFO:teuthology.orchestra.run.trial186.stdout:Fetched 702 MB in 3min 30s (3347 kB/s) 2026-02-23T08:11:39.729 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-23T08:11:39.887 INFO:teuthology.orchestra.run.trial186.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-23T08:11:39.889 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-23T08:11:39.891 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-23T08:11:39.913 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-23T08:11:39.925 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-23T08:11:39.926 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-23T08:11:39.942 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-23T08:11:39.953 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-23T08:11:39.954 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-23T08:11:39.975 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-23T08:11:39.983 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-23T08:11:39.988 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-23T08:11:40.023 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-23T08:11:40.034 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-23T08:11:40.035 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-23T08:11:40.052 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-23T08:11:40.061 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-23T08:11:40.062 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-23T08:11:40.081 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-23T08:11:40.089 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-23T08:11:40.090 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-23T08:11:40.106 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package librados2. 2026-02-23T08:11:40.114 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../07-librados2_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.115 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking librados2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.157 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libnbd0. 2026-02-23T08:11:40.165 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-23T08:11:40.165 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-23T08:11:40.180 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package librbd1. 2026-02-23T08:11:40.188 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../09-librbd1_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.189 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking librbd1 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.226 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libcephfs2. 2026-02-23T08:11:40.234 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.235 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libcephfs2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.255 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-rados. 2026-02-23T08:11:40.264 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.264 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-rados (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.281 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-23T08:11:40.289 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:40.290 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-ceph-argparse (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.303 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-cephfs. 2026-02-23T08:11:40.312 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.312 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-cephfs (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.329 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-23T08:11:40.337 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:40.337 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-ceph-common (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.356 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-23T08:11:40.364 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-23T08:11:40.364 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-23T08:11:40.380 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-prettytable. 2026-02-23T08:11:40.388 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-23T08:11:40.389 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-23T08:11:40.404 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-rbd. 2026-02-23T08:11:40.412 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.412 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-rbd (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.430 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-23T08:11:40.438 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-23T08:11:40.438 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-23T08:11:40.457 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package librgw2. 2026-02-23T08:11:40.465 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../19-librgw2_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.466 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking librgw2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.562 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-rgw. 2026-02-23T08:11:40.570 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.570 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-rgw (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.586 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-23T08:11:40.594 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-23T08:11:40.595 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-23T08:11:40.609 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libradosstriper1. 2026-02-23T08:11:40.618 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.618 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libradosstriper1 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.635 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-common. 2026-02-23T08:11:40.642 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.643 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-common (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.900 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-base. 2026-02-23T08:11:40.908 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:40.913 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-base (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:40.971 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-23T08:11:40.979 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-23T08:11:40.980 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-23T08:11:40.995 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-cheroot. 2026-02-23T08:11:41.003 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-23T08:11:41.003 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-23T08:11:41.020 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-23T08:11:41.029 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-23T08:11:41.030 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-23T08:11:41.044 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-23T08:11:41.052 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-23T08:11:41.052 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-23T08:11:41.067 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-23T08:11:41.075 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-23T08:11:41.075 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-23T08:11:41.090 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-tempora. 2026-02-23T08:11:41.099 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-23T08:11:41.099 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-23T08:11:41.114 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-portend. 2026-02-23T08:11:41.122 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-23T08:11:41.122 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-23T08:11:41.137 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-23T08:11:41.145 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-23T08:11:41.146 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-23T08:11:41.160 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-23T08:11:41.169 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-23T08:11:41.169 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-23T08:11:41.194 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-natsort. 2026-02-23T08:11:41.202 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-23T08:11:41.203 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-23T08:11:41.220 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-23T08:11:41.229 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:41.229 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:41.260 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-23T08:11:41.268 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:41.268 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:41.284 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mgr. 2026-02-23T08:11:41.292 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:41.293 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mgr (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:41.314 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mon. 2026-02-23T08:11:41.322 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:41.323 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mon (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:41.387 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-23T08:11:41.395 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-23T08:11:41.396 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-23T08:11:41.412 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-23T08:11:41.420 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:41.420 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-osd-classic (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:45.513 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-osd. 2026-02-23T08:11:45.526 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:45.526 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-osd (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:45.546 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph. 2026-02-23T08:11:45.559 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../42-ceph_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:45.559 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:45.575 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-fuse. 2026-02-23T08:11:45.588 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:45.588 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-fuse (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:45.612 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mds. 2026-02-23T08:11:45.624 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:45.624 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mds (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:45.657 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package cephadm. 2026-02-23T08:11:45.666 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../45-cephadm_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:45.666 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking cephadm (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:45.683 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-23T08:11:45.691 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-23T08:11:45.691 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-23T08:11:45.713 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-23T08:11:45.721 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:45.722 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:45.743 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-23T08:11:45.751 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-23T08:11:45.752 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-23T08:11:45.767 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-routes. 2026-02-23T08:11:45.775 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-23T08:11:45.775 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-23T08:11:45.796 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-23T08:11:45.804 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:45.804 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:46.489 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-23T08:11:46.500 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-23T08:11:46.500 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-23T08:11:46.539 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-joblib. 2026-02-23T08:11:46.548 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-23T08:11:46.548 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-23T08:11:46.576 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-23T08:11:46.585 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-23T08:11:46.585 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-23T08:11:46.600 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-sklearn. 2026-02-23T08:11:46.608 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-23T08:11:46.609 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-23T08:11:46.694 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-23T08:11:46.704 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:46.704 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:46.860 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-cachetools. 2026-02-23T08:11:46.869 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-23T08:11:46.869 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-23T08:11:46.884 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-rsa. 2026-02-23T08:11:46.893 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-23T08:11:46.893 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-23T08:11:46.912 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-google-auth. 2026-02-23T08:11:46.920 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-23T08:11:46.920 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-23T08:11:46.939 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-23T08:11:46.948 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-23T08:11:46.948 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-23T08:11:46.964 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-websocket. 2026-02-23T08:11:46.972 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-23T08:11:46.972 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-23T08:11:46.990 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-23T08:11:46.999 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-23T08:11:46.999 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-23T08:11:47.094 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-23T08:11:47.102 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:47.103 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:47.117 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-23T08:11:47.126 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-23T08:11:47.126 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-23T08:11:47.142 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-23T08:11:47.150 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-23T08:11:47.151 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-23T08:11:47.166 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package jq. 2026-02-23T08:11:47.175 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-23T08:11:47.175 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-23T08:11:47.189 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package socat. 2026-02-23T08:11:47.198 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-23T08:11:47.198 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-23T08:11:47.219 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package xmlstarlet. 2026-02-23T08:11:47.227 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-23T08:11:47.228 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-23T08:11:47.263 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-test. 2026-02-23T08:11:47.272 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:47.272 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-test (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:47.911 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package ceph-volume. 2026-02-23T08:11:47.920 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5433-g5adce3ff-1jammy_all.deb ... 2026-02-23T08:11:47.920 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking ceph-volume (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:47.945 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-23T08:11:47.954 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:47.955 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libcephfs-daemon (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:47.969 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-23T08:11:47.978 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:47.978 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libcephfs-proxy2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:47.993 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-23T08:11:48.002 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:48.002 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking libcephfs-dev (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:48.019 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package nvme-cli. 2026-02-23T08:11:48.027 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-23T08:11:48.027 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-23T08:11:48.059 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-23T08:11:48.068 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-23T08:11:48.069 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-23T08:11:48.099 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-23T08:11:48.107 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-23T08:11:48.108 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-23T08:11:48.124 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-pluggy. 2026-02-23T08:11:48.132 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-23T08:11:48.133 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-23T08:11:48.149 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-psutil. 2026-02-23T08:11:48.157 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-23T08:11:48.158 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-23T08:11:48.177 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-py. 2026-02-23T08:11:48.186 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-23T08:11:48.186 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-23T08:11:48.207 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-pygments. 2026-02-23T08:11:48.215 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-23T08:11:48.216 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-23T08:11:48.257 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-toml. 2026-02-23T08:11:48.266 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-23T08:11:48.266 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-23T08:11:48.282 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-pytest. 2026-02-23T08:11:48.290 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-23T08:11:48.290 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-23T08:11:48.312 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-simplejson. 2026-02-23T08:11:48.321 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-23T08:11:48.321 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-23T08:11:48.346 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package python3-webob. 2026-02-23T08:11:48.355 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-23T08:11:48.355 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-23T08:11:48.372 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-23T08:11:48.381 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-23T08:11:48.381 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-23T08:11:48.449 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package radosgw. 2026-02-23T08:11:48.458 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../85-radosgw_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:48.458 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking radosgw (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:48.626 INFO:teuthology.orchestra.run.trial186.stdout:Selecting previously unselected package rbd-fuse. 2026-02-23T08:11:48.634 INFO:teuthology.orchestra.run.trial186.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5433-g5adce3ff-1jammy_amd64.deb ... 2026-02-23T08:11:48.635 INFO:teuthology.orchestra.run.trial186.stdout:Unpacking rbd-fuse (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:48.662 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-23T08:11:48.728 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-23T08:11:48.729 INFO:teuthology.orchestra.run.trial186.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-23T08:11:48.801 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-23T08:11:49.073 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-23T08:11:49.441 INFO:teuthology.orchestra.run.trial186.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-23T08:11:49.451 INFO:teuthology.orchestra.run.trial186.stdout:Setting up cephadm (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:49.495 INFO:teuthology.orchestra.run.trial186.stdout:Adding system user cephadm....done 2026-02-23T08:11:49.501 INFO:teuthology.orchestra.run.trial186.stdout:usermod: no changes 2026-02-23T08:11:49.506 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-23T08:11:49.577 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-23T08:11:49.577 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-23T08:11:49.648 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-23T08:11:49.724 INFO:teuthology.orchestra.run.trial186.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-23T08:11:49.725 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-23T08:11:49.820 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-23T08:11:49.943 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-23T08:11:50.018 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-23T08:11:50.079 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-ceph-argparse (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:50.146 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-23T08:11:50.147 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-23T08:11:50.148 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-23T08:11:50.149 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-23T08:11:50.149 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-23T08:11:50.273 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-23T08:11:50.350 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libcephfs-proxy2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:50.351 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-23T08:11:50.426 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-23T08:11:50.504 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-23T08:11:50.751 INFO:teuthology.orchestra.run.trial186.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-23T08:11:50.752 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-23T08:11:50.846 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-23T08:11:50.969 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-23T08:11:51.050 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-23T08:11:51.121 INFO:teuthology.orchestra.run.trial186.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-23T08:11:51.122 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-ceph-common (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:51.212 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-23T08:11:51.706 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-23T08:11:51.711 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-23T08:11:51.789 INFO:teuthology.orchestra.run.trial186.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-23T08:11:51.790 INFO:teuthology.orchestra.run.trial186.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-23T08:11:51.791 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-23T08:11:51.867 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-23T08:11:51.940 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-23T08:11:51.941 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-23T08:11:52.018 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-23T08:11:52.090 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-23T08:11:52.166 INFO:teuthology.orchestra.run.trial186.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-23T08:11:52.167 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-23T08:11:52.247 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-23T08:11:52.248 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-23T08:11:52.324 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-23T08:11:52.408 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-23T08:11:52.482 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-23T08:11:52.483 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-23T08:11:52.619 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-23T08:11:52.692 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-23T08:11:52.693 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-23T08:11:52.774 INFO:teuthology.orchestra.run.trial186.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-23T08:11:52.775 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-23T08:11:52.909 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-23T08:11:52.911 INFO:teuthology.orchestra.run.trial186.stdout:Setting up librados2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:52.911 INFO:teuthology.orchestra.run.trial186.stdout:Setting up librgw2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:52.912 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:52.913 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-23T08:11:53.427 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libcephfs2 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.428 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libradosstriper1 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.429 INFO:teuthology.orchestra.run.trial186.stdout:Setting up librbd1 (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.430 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mgr-modules-core (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.430 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-fuse (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.505 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-23T08:11:53.506 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-23T08:11:53.890 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libcephfs-dev (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.891 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-rados (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.892 INFO:teuthology.orchestra.run.trial186.stdout:Setting up libcephfs-daemon (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.893 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-rbd (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.893 INFO:teuthology.orchestra.run.trial186.stdout:Setting up rbd-fuse (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.894 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-rgw (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.895 INFO:teuthology.orchestra.run.trial186.stdout:Setting up python3-cephfs (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.896 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-common (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:53.925 INFO:teuthology.orchestra.run.trial186.stdout:Adding group ceph....done 2026-02-23T08:11:53.960 INFO:teuthology.orchestra.run.trial186.stdout:Adding system user ceph....done 2026-02-23T08:11:53.970 INFO:teuthology.orchestra.run.trial186.stdout:Setting system user ceph properties....done 2026-02-23T08:11:53.977 INFO:teuthology.orchestra.run.trial186.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-23T08:11:54.053 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-23T08:11:54.323 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-23T08:11:54.669 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-test (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:54.670 INFO:teuthology.orchestra.run.trial186.stdout:Setting up radosgw (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:54.929 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-23T08:11:54.929 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-23T08:11:55.290 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-base (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:55.406 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-23T08:11:55.779 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mds (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:55.839 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-23T08:11:55.839 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-23T08:11:56.213 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mgr (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:56.288 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-23T08:11:56.288 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-23T08:11:56.654 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:56.655 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-osd-classic (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:56.672 INFO:teuthology.orchestra.run.trial186.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-23T08:11:56.681 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:56.692 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mon (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:56.765 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-23T08:11:56.765 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-23T08:11:57.139 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mgr-cephadm (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:57.149 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-mgr-dashboard (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:57.160 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-osd (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:57.251 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-23T08:11:57.251 INFO:teuthology.orchestra.run.trial186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-23T08:11:57.642 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:57.643 INFO:teuthology.orchestra.run.trial186.stdout:Setting up ceph-volume (20.3.0-5433-g5adce3ff-1jammy) ... 2026-02-23T08:11:57.750 INFO:teuthology.orchestra.run.trial186.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-23T08:11:57.833 INFO:teuthology.orchestra.run.trial186.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-23T08:11:58.464 INFO:teuthology.orchestra.run.trial186.stdout: 2026-02-23T08:11:58.464 INFO:teuthology.orchestra.run.trial186.stdout:Running kernel seems to be up-to-date. 2026-02-23T08:11:58.464 INFO:teuthology.orchestra.run.trial186.stdout: 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout:The processor microcode seems to be up-to-date. 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout: 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout:No services need to be restarted. 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout: 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout:No containers need to be restarted. 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout: 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout:No user sessions are running outdated binaries. 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout: 2026-02-23T08:11:58.465 INFO:teuthology.orchestra.run.trial186.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-23T08:11:59.256 INFO:teuthology.orchestra.run.trial186.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-23T08:11:59.264 DEBUG:teuthology.parallel:result is None 2026-02-23T08:11:59.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=5adce3ff0fe1645dcbc38d059f948571dc3d69da 2026-02-23T08:11:59.392 DEBUG:teuthology.orchestra.run.trial186:> dpkg-query -W -f '${Version}' ceph 2026-02-23T08:11:59.403 INFO:teuthology.orchestra.run.trial186.stdout:20.3.0-5433-g5adce3ff-1jammy 2026-02-23T08:11:59.403 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5433-g5adce3ff-1jammy 2026-02-23T08:11:59.403 INFO:teuthology.task.install:The correct ceph version 20.3.0-5433-g5adce3ff-1jammy is installed. 2026-02-23T08:11:59.406 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-23T08:11:59.406 DEBUG:teuthology.orchestra.run.trial186:> set -ex 2026-02-23T08:11:59.406 DEBUG:teuthology.orchestra.run.trial186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-23T08:11:59.452 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-23T08:11:59.452 DEBUG:teuthology.orchestra.run.trial186:> set -ex 2026-02-23T08:11:59.452 DEBUG:teuthology.orchestra.run.trial186:> sudo dd of=/usr/bin/daemon-helper 2026-02-23T08:11:59.505 DEBUG:teuthology.orchestra.run.trial186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-23T08:11:59.555 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-23T08:11:59.556 DEBUG:teuthology.orchestra.run.trial186:> set -ex 2026-02-23T08:11:59.556 DEBUG:teuthology.orchestra.run.trial186:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-23T08:11:59.605 DEBUG:teuthology.orchestra.run.trial186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-23T08:11:59.655 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-23T08:11:59.656 DEBUG:teuthology.orchestra.run.trial186:> set -ex 2026-02-23T08:11:59.656 DEBUG:teuthology.orchestra.run.trial186:> sudo dd of=/usr/bin/stdin-killer 2026-02-23T08:11:59.705 DEBUG:teuthology.orchestra.run.trial186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-23T08:11:59.756 INFO:teuthology.run_tasks:Running task workunit... 2026-02-23T08:11:59.784 INFO:tasks.workunit:Pulling workunits from ref 5adce3ff0fe1645dcbc38d059f948571dc3d69da 2026-02-23T08:11:59.785 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-23T08:11:59.785 INFO:tasks.workunit:timeout=3h 2026-02-23T08:11:59.785 INFO:tasks.workunit:cleanup=True 2026-02-23T08:11:59.785 DEBUG:teuthology.orchestra.run.trial186:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-23T08:11:59.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T08:11:59.800 INFO:teuthology.orchestra.run.trial186.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-23T08:11:59.800 DEBUG:teuthology.orchestra.run.trial186:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-23T08:11:59.847 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-23T08:11:59.848 DEBUG:teuthology.orchestra.run.trial186:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-23T08:11:59.891 DEBUG:teuthology.orchestra.run.trial186:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5adce3ff0fe1645dcbc38d059f948571dc3d69da 2026-02-23T08:11:59.942 INFO:tasks.workunit.client.0.trial186.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-23T08:12:21.968 INFO:tasks.workunit.client.0.trial186.stderr:Note: switching to '5adce3ff0fe1645dcbc38d059f948571dc3d69da'. 2026-02-23T08:12:21.968 INFO:tasks.workunit.client.0.trial186.stderr: 2026-02-23T08:12:21.968 INFO:tasks.workunit.client.0.trial186.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-23T08:12:21.968 INFO:tasks.workunit.client.0.trial186.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-23T08:12:21.968 INFO:tasks.workunit.client.0.trial186.stderr:state without impacting any branches by switching back to a branch. 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: git switch -c 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr:Or undo this operation with: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: git switch - 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr: 2026-02-23T08:12:21.969 INFO:tasks.workunit.client.0.trial186.stderr:HEAD is now at 5adce3ff0fe Merge branch 'wip-qa-distros-t-rocky' of https://github.com/cbodley/ceph into wip-bharath8-testing-2026-02-22-0716 2026-02-23T08:12:21.978 DEBUG:teuthology.orchestra.run.trial186:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-23T08:12:22.025 DEBUG:teuthology.orchestra.run.trial186:> set -ex 2026-02-23T08:12:22.025 DEBUG:teuthology.orchestra.run.trial186:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-23T08:12:22.072 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-23T08:12:22.072 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-23T08:12:22.072 DEBUG:teuthology.orchestra.run.trial186:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5adce3ff0fe1645dcbc38d059f948571dc3d69da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-23T08:12:22.119 INFO:tasks.workunit.client.0.trial186.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-23T08:12:22.119 INFO:tasks.workunit.client.0.trial186.stderr:++ TIMEOUT=300 2026-02-23T08:12:22.119 INFO:tasks.workunit.client.0.trial186.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-23T08:12:22.119 INFO:tasks.workunit.client.0.trial186.stderr:++ MAX_TIMEOUT=15 2026-02-23T08:12:22.119 INFO:tasks.workunit.client.0.trial186.stderr:++ PG_NUM=4 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:++ TMPDIR=/tmp 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:+++ uname 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:++ '[' Linux = FreeBSD ']' 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:++ SED=sed 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:++ AWK=awk 2026-02-23T08:12:22.120 INFO:tasks.workunit.client.0.trial186.stderr:+++ stty -a 2026-02-23T08:12:22.121 INFO:tasks.workunit.client.0.trial186.stderr:+++ head -1 2026-02-23T08:12:22.121 INFO:tasks.workunit.client.0.trial186.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-23T08:12:22.122 INFO:tasks.workunit.client.0.trial186.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-23T08:12:22.122 INFO:tasks.workunit.client.0.trial186.stderr:++ termwidth= 2026-02-23T08:12:22.122 INFO:tasks.workunit.client.0.trial186.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-23T08:12:22.122 INFO:tasks.workunit.client.0.trial186.stderr:++ DIFFCOLOPTS='-y ' 2026-02-23T08:12:22.122 INFO:tasks.workunit.client.0.trial186.stderr:++ KERNCORE=kernel.core_pattern 2026-02-23T08:12:22.122 INFO:tasks.workunit.client.0.trial186.stderr:++ EXTRA_OPTS= 2026-02-23T08:12:22.125 INFO:tasks.workunit.client.0.trial186.stderr:++ test '' = TESTS 2026-02-23T08:12:22.125 INFO:tasks.workunit.client.0.trial186.stderr:++ uname -m 2026-02-23T08:12:22.126 INFO:tasks.workunit.client.0.trial186.stderr:+ arch=x86_64 2026-02-23T08:12:22.126 INFO:tasks.workunit.client.0.trial186.stderr:+ case $arch in 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ main test-erasure-code-plugins 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ shift 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ shopt -s -o xtrace 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-23T08:12:22.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code-plugins 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-23T08:12:22.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none ' 2026-02-23T08:12:22.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-23T08:12:22.129 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-23T08:12:22.129 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-23T08:12:22.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_blaum_roth_warning 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:TEST_ec_profile_warning 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:TEST_preload_no_warning 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:TEST_preload_no_warning_default 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:TEST_preload_warning' 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_blaum_roth_warning td/test-erasure-code-plugins 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:119: TEST_ec_profile_blaum_roth_warning: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:121: TEST_ec_profile_blaum_roth_warning: setup td/test-erasure-code-plugins 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:12:22.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:12:22.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:12:22.134 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:12:22.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:12:22.135 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:12:22.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:12:22.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:12:22.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:12:22.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:12:22.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:12:22.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:12:22.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:12:22.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:12:22.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:12:22.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:12:22.142 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:12:22.142 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:22.142 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:22.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:12:22.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:12:22.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:12:22.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:12:22.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:12:22.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:22.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:22.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:12:22.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:122: TEST_ec_profile_blaum_roth_warning: run_mon td/test-erasure-code-plugins a 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:12:22.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.184 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:12:22.185 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:22.185 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:22.185 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:22.185 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:22.185 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:22.185 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:22.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:12:22.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:12:22.217 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:22.218 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:22.219 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:12:22.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:12:22.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:12:22.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:12:22.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:12:22.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:12:22.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:22.277 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:12:22.278 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:12:22.278 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:12:22.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:123: TEST_ec_profile_blaum_roth_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:12:22.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:12:22.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:12:22.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:12:22.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:12:22.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:12:22.444 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:12:22.444 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:22.444 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:22.445 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:22.445 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:22.445 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:22.445 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:22.445 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:12:22.446 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:12:22.464 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: seq 0 2 2026-02-23T08:12:22.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-23T08:12:22.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 0 2026-02-23T08:12:22.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:22.466 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:12:22.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:12:22.468 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:12:22.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9a33698d-d8d4-4b31-a14d-fc0975d20206 2026-02-23T08:12:22.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9a33698d-d8d4-4b31-a14d-fc0975d20206' 2026-02-23T08:12:22.469 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 9a33698d-d8d4-4b31-a14d-fc0975d20206 2026-02-23T08:12:22.469 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:12:22.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDmC5xpHfWCHBAAaC+qucNBXbsBG9DUGX2o0A== 2026-02-23T08:12:22.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDmC5xpHfWCHBAAaC+qucNBXbsBG9DUGX2o0A=="}' 2026-02-23T08:12:22.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9a33698d-d8d4-4b31-a14d-fc0975d20206 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:12:22.585 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:22.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:12:22.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDmC5xpHfWCHBAAaC+qucNBXbsBG9DUGX2o0A== --osd-uuid 9a33698d-d8d4-4b31-a14d-fc0975d20206 2026-02-23T08:12:22.608 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:22.603+0000 7f2855eafa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:22.609 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:22.607+0000 7f2855eafa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:22.610 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:22.607+0000 7f2855eafa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:22.611 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:22.607+0000 7f2855eafa40 -1 bdev(0x55b743d56400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:12:22.611 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:22.607+0000 7f2855eafa40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:12:23.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:12:23.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:12:23.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:12:23.296 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:12:23.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:12:23.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:12:23.414 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:12:23.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:12:23.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:12:23.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:12:23.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:12:23.435 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:23.431+0000 7f7e306dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:23.437 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:23.435+0000 7f7e306dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:23.438 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:23.435+0000 7f7e306dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:23.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:12:23.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:12:23.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:12:23.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:12:23.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:12:23.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:23.523 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:23.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:12:23.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:23.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:12:23.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:23.818 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:23.815+0000 7f7e306dfa40 -1 Falling back to public interface 2026-02-23T08:12:24.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:24.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:24.630 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:12:24.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:12:24.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:24.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:12:24.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:24.858 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:24.855+0000 7f7e306dfa40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:12:25.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:25.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:25.734 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:12:25.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:12:25.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:25.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:12:25.941 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2459384490,v1:127.0.0.1:6801/2459384490] [v2:127.0.0.1:6802/2459384490,v1:127.0.0.1:6803/2459384490] exists,up 9a33698d-d8d4-4b31-a14d-fc0975d20206 2026-02-23T08:12:25.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:12:25.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:12:25.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 1 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:25.944 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:12:25.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a30761a1-aaa7-4413-ae29-8b18f46027ce 2026-02-23T08:12:25.946 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 a30761a1-aaa7-4413-ae29-8b18f46027ce 2026-02-23T08:12:25.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a30761a1-aaa7-4413-ae29-8b18f46027ce' 2026-02-23T08:12:25.946 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:12:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDpC5xp28MKORAAfFs8mQz0BU7TQMkuBog0mA== 2026-02-23T08:12:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDpC5xp28MKORAAfFs8mQz0BU7TQMkuBog0mA=="}' 2026-02-23T08:12:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a30761a1-aaa7-4413-ae29-8b18f46027ce -i td/test-erasure-code-plugins/1/new.json 2026-02-23T08:12:26.181 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:12:26.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-23T08:12:26.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDpC5xp28MKORAAfFs8mQz0BU7TQMkuBog0mA== --osd-uuid a30761a1-aaa7-4413-ae29-8b18f46027ce 2026-02-23T08:12:26.205 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:26.203+0000 7f83d4189a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:26.207 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:26.203+0000 7f83d4189a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:26.208 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:26.207+0000 7f83d4189a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:26.208 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:26.207+0000 7f83d4189a40 -1 bdev(0x5572bad72400 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:12:26.209 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:26.207+0000 7f83d4189a40 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-23T08:12:26.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-23T08:12:26.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:12:26.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:12:26.852 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:12:26.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:12:27.139 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:12:27.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:12:27.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:12:27.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:12:27.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:12:27.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:12:27.162 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:27.159+0000 7f880eda6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:27.163 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:27.159+0000 7f880eda6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:27.165 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:27.163+0000 7f880eda6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:27.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:12:27.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:12:27.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:12:27.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:12:27.362 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:27.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:12:27.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:27.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:12:27.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:27.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:12:27.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:27.581 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:27.579+0000 7f880eda6a40 -1 Falling back to public interface 2026-02-23T08:12:28.121 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:28.119+0000 7f880eda6a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:12:28.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:28.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:28.576 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:12:28.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:12:28.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:28.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:12:28.792 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4083469973,v1:127.0.0.1:6811/4083469973] [v2:127.0.0.1:6812/4083469973,v1:127.0.0.1:6813/4083469973] exists,up a30761a1-aaa7-4413-ae29-8b18f46027ce 2026-02-23T08:12:28.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:12:28.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:12:28.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:12:28.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-23T08:12:28.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 2 2026-02-23T08:12:28.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:28.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:12:28.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:12:28.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:12:28.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-23T08:12:28.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:12:28.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:12:28.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:12:28.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:12:28.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-23T08:12:28.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-23T08:12:28.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:12:28.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:12:28.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:12:28.802 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:12:28.802 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:28.803 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:28.803 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:28.803 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:28.803 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:28.804 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:28.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:28.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:12:28.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:12:28.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:12:28.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:12:28.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:12:28.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:12:28.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fa85bdb1-ff4b-45cf-a1fc-6277b9d9abd3 2026-02-23T08:12:28.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 fa85bdb1-ff4b-45cf-a1fc-6277b9d9abd3' 2026-02-23T08:12:28.807 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 fa85bdb1-ff4b-45cf-a1fc-6277b9d9abd3 2026-02-23T08:12:28.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:12:28.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDsC5xpcyIpMBAAt9X7POAPliujXR4sLGu6Rw== 2026-02-23T08:12:28.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDsC5xpcyIpMBAAt9X7POAPliujXR4sLGu6Rw=="}' 2026-02-23T08:12:28.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fa85bdb1-ff4b-45cf-a1fc-6277b9d9abd3 -i td/test-erasure-code-plugins/2/new.json 2026-02-23T08:12:29.032 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:12:29.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-23T08:12:29.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDsC5xpcyIpMBAAt9X7POAPliujXR4sLGu6Rw== --osd-uuid fa85bdb1-ff4b-45cf-a1fc-6277b9d9abd3 2026-02-23T08:12:29.057 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:29.055+0000 7fa5dd425a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:29.058 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:29.055+0000 7fa5dd425a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:29.059 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:29.055+0000 7fa5dd425a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:29.060 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:29.055+0000 7fa5dd425a40 -1 bdev(0x56086d722400 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:12:29.060 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:29.059+0000 7fa5dd425a40 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-23T08:12:29.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-23T08:12:29.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:12:29.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:12:29.693 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:12:29.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:12:29.987 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:12:29.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:12:29.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:12:29.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:12:29.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:12:29.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:12:30.002 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:29.999+0000 7f88e3437a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:30.003 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:29.999+0000 7f88e3437a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:30.004 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:30.003+0000 7f88e3437a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:30.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:12:30.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:12:30.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:12:30.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:12:30.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:12:30.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:30.204 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:30.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:12:30.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:30.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:12:30.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:30.434 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:30.431+0000 7f88e3437a40 -1 Falling back to public interface 2026-02-23T08:12:31.033 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:31.031+0000 7f88e3437a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:12:31.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:31.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:31.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:12:31.418 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:12:31.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:31.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:12:31.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:32.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:32.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:32.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:12:32.633 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:12:32.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:32.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:12:32.849 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1586716958,v1:127.0.0.1:6819/1586716958] [v2:127.0.0.1:6820/1586716958,v1:127.0.0.1:6821/1586716958] exists,up fa85bdb1-ff4b-45cf-a1fc-6277b9d9abd3 2026-02-23T08:12:32.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:12:32.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:12:32.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:12:32.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:127: TEST_ec_profile_blaum_roth_warning: create_rbd_pool 2026-02-23T08:12:32.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-23T08:12:33.058 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' does not exist 2026-02-23T08:12:33.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-23T08:12:33.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:12:33.342 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:12:33.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:12:34.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-23T08:12:34.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:128: TEST_ec_profile_blaum_roth_warning: wait_for_clean 2026-02-23T08:12:34.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:12:34.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:12:34.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:12:34.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:12:34.636 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:12:34.636 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:12:34.636 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:12:34.636 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:12:34.636 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:12:34.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:12:34.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:12:34.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:12:34.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:12:34.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:12:34.745 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:12:34.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:12:34.959 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:12:34.959 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:12:34.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:12:34.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:12:34.960 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:12:35.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-23T08:12:35.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-23T08:12:35.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-23T08:12:35.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:12:35.037 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:12:35.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-23T08:12:35.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-23T08:12:35.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-23T08:12:35.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:12:35.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:12:35.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-23T08:12:35.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-23T08:12:35.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964 2-64424509443' 2026-02-23T08:12:35.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:12:35.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-23T08:12:35.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:12:35.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:12:35.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-23T08:12:35.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:12:35.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-23T08:12:35.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-23T08:12:35.197 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 21474836484 2026-02-23T08:12:35.197 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:12:35.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-23T08:12:35.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:12:35.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-23T08:12:35.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:12:35.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:12:35.416 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-23T08:12:35.416 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:12:35.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-23T08:12:35.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-23T08:12:35.417 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 42949672964 2026-02-23T08:12:35.418 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:12:35.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-23T08:12:35.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:12:35.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-64424509443 2026-02-23T08:12:35.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:12:35.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:12:35.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509443 2026-02-23T08:12:35.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:12:35.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509443 2026-02-23T08:12:35.649 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 64424509443 2026-02-23T08:12:35.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-23T08:12:35.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:12:35.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509441 -lt 64424509443 2026-02-23T08:12:35.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:12:36.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:12:36.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:12:37.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509441 -lt 64424509443 2026-02-23T08:12:37.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:12:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-23T08:12:38.086 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:12:38.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-23T08:12:38.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:12:38.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:12:38.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:12:38.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-23T08:12:38.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:12:38.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:12:38.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:12:38.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:12:38.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:12:38.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:12:38.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:12:38.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-23T08:12:38.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:12:38.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:12:38.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:12:39.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-23T08:12:39.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:12:39.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:12:39.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:130: TEST_ec_profile_blaum_roth_warning: echo 'Starting test for blaum-roth profile health warning' 2026-02-23T08:12:39.100 INFO:tasks.workunit.client.0.trial186.stdout:Starting test for blaum-roth profile health warning 2026-02-23T08:12:39.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:133: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set prof- plugin=jerasure k=3 m=1 technique=blaum_roth w=7 --yes-i-really-mean-it --force 2026-02-23T08:12:39.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:135: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set test crush-device-class= crush-failure-domain=osd crush-num-failure-domains=0 crush-osds-per-failure-domain=0 crush-root=default k=4 l=3 m=2 name=lrcprofile plugin=lrc 2026-02-23T08:12:39.724 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: get_asok_path mon.a 2026-02-23T08:12:39.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:12:39.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:12:39.725 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:12:39.725 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:39.725 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:39.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:12:39.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: CEPH_ARGS= 2026-02-23T08:12:39.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:12:39.775 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:12:39.781 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:137: TEST_ec_profile_blaum_roth_warning: sleep 10 2026-02-23T08:12:49.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:138: TEST_ec_profile_blaum_roth_warning: grep -F '1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:12:42.211+0000 7f0858151640 0 log_channel(cluster) log [WRN] : Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:12:42.211+0000 7f085594c640 10 mon.a@0(leader).log v89 logging 2026-02-23T08:12:42.215057+0000 mon.a (mon.0) 155 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:12:42.263+0000 7f0853147640 7 mon.a@0(leader).log v90 update_from_paxos applying incremental log 90 2026-02-23T08:12:42.215057+0000 mon.a (mon.0) 155 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.786 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.787 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.787 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.787 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.787 INFO:tasks.workunit.client.0.trial186.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-23T08:12:49.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:139: TEST_ec_profile_blaum_roth_warning: grep -F 'w+1=8 for the EC profile prof- is not prime and could lead to data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:12:49.789 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.789 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.789 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.789 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.789 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.790 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.790 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.790 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.790 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.790 INFO:tasks.workunit.client.0.trial186.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:141: TEST_ec_profile_blaum_roth_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:12:49.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:12:49.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:12:49.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:12:49.898 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:12:49.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:12:49.899 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:12:49.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:12:49.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:12:49.901 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:12:49.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:12:49.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:12:49.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:12:49.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:12:49.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:12:49.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:12:49.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:12:49.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:12:49.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:49.913 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:49.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:12:49.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:12:49.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:12:49.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:142: TEST_ec_profile_blaum_roth_warning: return 0 2026-02-23T08:12:49.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-23T08:12:49.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:12:49.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:12:49.917 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:12:49.917 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:12:49.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:12:49.918 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:12:49.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:12:49.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:12:49.920 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:12:49.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:12:49.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:12:49.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:12:49.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:12:49.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:12:49.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:12:49.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:12:49.924 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:12:49.924 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:49.924 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:49.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:12:49.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:12:49.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:12:49.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:12:49.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:12:49.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:49.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:49.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:12:49.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:12:49.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-23T08:12:49.949 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:12:49.950 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:49.950 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:49.950 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:49.951 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:49.951 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:49.951 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:49.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:12:49.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:12:49.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:12:49.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:12:49.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:12:49.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:12:49.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:12:49.982 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:12:50.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:12:50.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:12:50.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:12:50.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:12:50.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:12:50.056 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:12:50.056 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:12:50.056 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:12:50.056 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:12:50.056 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:50.056 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:50.057 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:12:50.057 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:12:50.057 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:12:50.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:12:50.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:12:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:12:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:12:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:12:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:12:50.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:12:50.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:50.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:50.220 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:50.220 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:50.220 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:50.220 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:50.220 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:12:50.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:12:50.238 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-23T08:12:50.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-23T08:12:50.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-23T08:12:50.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:50.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:12:50.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:12:50.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:12:50.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:12:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:12:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:12:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:12:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:12:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:12:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:12:50.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:12:50.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:12:50.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:12:50.247 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:12:50.247 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:50.247 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:50.247 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:50.248 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:50.248 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:50.248 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:50.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:50.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:12:50.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:12:50.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:12:50.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:12:50.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:12:50.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:12:50.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:12:50.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:12:50.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:12:50.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:12:50.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:12:50.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:12:50.250 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:12:50.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d36d0ffc-56b9-41b4-bcc1-8e0354b943ae 2026-02-23T08:12:50.251 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 d36d0ffc-56b9-41b4-bcc1-8e0354b943ae 2026-02-23T08:12:50.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d36d0ffc-56b9-41b4-bcc1-8e0354b943ae' 2026-02-23T08:12:50.252 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:12:50.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQACDJxpiZ8XDxAAZaxO3owY55xiFJ79CdZVig== 2026-02-23T08:12:50.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQACDJxpiZ8XDxAAZaxO3owY55xiFJ79CdZVig=="}' 2026-02-23T08:12:50.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d36d0ffc-56b9-41b4-bcc1-8e0354b943ae -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:12:50.363 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:50.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:12:50.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQACDJxpiZ8XDxAAZaxO3owY55xiFJ79CdZVig== --osd-uuid d36d0ffc-56b9-41b4-bcc1-8e0354b943ae 2026-02-23T08:12:50.390 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:50.387+0000 7f105a7e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:50.391 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:50.387+0000 7f105a7e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:50.392 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:50.391+0000 7f105a7e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:50.393 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:50.391+0000 7f105a7e0a40 -1 bdev(0x563a858c0400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:12:50.393 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:50.391+0000 7f105a7e0a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:12:51.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:12:51.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:12:51.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:12:51.149 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:12:51.149 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:12:51.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:12:51.267 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:12:51.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:12:51.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:12:51.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:12:51.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:12:51.281 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:51.279+0000 7f6d7bc87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:51.282 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:51.279+0000 7f6d7bc87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:51.284 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:51.279+0000 7f6d7bc87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:51.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:12:51.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:12:51.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:12:51.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:12:51.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:12:51.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:51.389 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:51.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:12:51.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:51.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:12:51.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:51.701 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:51.699+0000 7f6d7bc87a40 -1 Falling back to public interface 2026-02-23T08:12:52.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:52.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:52.496 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:12:52.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:12:52.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:52.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:12:52.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:52.745 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:52.743+0000 7f6d7bc87a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:12:53.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:53.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:53.601 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:12:53.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:12:53.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:53.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:12:53.812 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4160477960,v1:127.0.0.1:6801/4160477960] [v2:127.0.0.1:6802/4160477960,v1:127.0.0.1:6803/4160477960] exists,up d36d0ffc-56b9-41b4-bcc1-8e0354b943ae 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:12:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:12:53.817 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:12:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:12:53.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:12:53.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:12:53.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:12:53.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:12:53.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-23T08:12:53.819 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:12:53.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4fd06c48-079a-40c6-a5b8-b32ca584d234 2026-02-23T08:12:53.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4fd06c48-079a-40c6-a5b8-b32ca584d234' 2026-02-23T08:12:53.820 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 4fd06c48-079a-40c6-a5b8-b32ca584d234 2026-02-23T08:12:53.820 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:12:53.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAFDJxpvF5kMRAAXmWioiYnnu5fL0fAK+/dzA== 2026-02-23T08:12:53.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAFDJxpvF5kMRAAXmWioiYnnu5fL0fAK+/dzA=="}' 2026-02-23T08:12:53.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4fd06c48-079a-40c6-a5b8-b32ca584d234 -i td/test-erasure-code-plugins/1/new.json 2026-02-23T08:12:54.047 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:12:54.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-23T08:12:54.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAFDJxpvF5kMRAAXmWioiYnnu5fL0fAK+/dzA== --osd-uuid 4fd06c48-079a-40c6-a5b8-b32ca584d234 2026-02-23T08:12:54.080 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:54.079+0000 7f02e857aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:54.081 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:54.079+0000 7f02e857aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:54.083 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:54.079+0000 7f02e857aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:54.083 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:54.079+0000 7f02e857aa40 -1 bdev(0x5648fa5a8400 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:12:54.083 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:54.079+0000 7f02e857aa40 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-23T08:12:54.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-23T08:12:54.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:12:54.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:12:54.745 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:12:54.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:12:55.024 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:12:55.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:12:55.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:12:55.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:12:55.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:12:55.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:12:55.046 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:55.043+0000 7fda867a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:55.048 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:55.047+0000 7fda867a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:55.049 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:55.047+0000 7fda867a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:55.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:12:55.409 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:55.407+0000 7fda867a3a40 -1 Falling back to public interface 2026-02-23T08:12:55.464 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:56.034 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:56.031+0000 7fda867a3a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:12:56.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:56.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:56.466 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:12:56.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:12:56.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:56.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:12:56.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:12:57.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:12:57.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:57.689 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:12:57.690 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:12:57.690 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:57.690 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:12:57.893 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2040966562,v1:127.0.0.1:6811/2040966562] [v2:127.0.0.1:6812/2040966562,v1:127.0.0.1:6813/2040966562] exists,up 4fd06c48-079a-40c6-a5b8-b32ca584d234 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:12:57.922 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b963bfde-d129-4e47-aff2-f14d7bf4e54d 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b963bfde-d129-4e47-aff2-f14d7bf4e54d' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 b963bfde-d129-4e47-aff2-f14d7bf4e54d 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAJDJxpmbUwNhAA2tMEaf+OxLKl1WUGaNfMQw== 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAJDJxpmbUwNhAA2tMEaf+OxLKl1WUGaNfMQw=="}' 2026-02-23T08:12:57.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b963bfde-d129-4e47-aff2-f14d7bf4e54d -i td/test-erasure-code-plugins/2/new.json 2026-02-23T08:12:58.127 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:12:58.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-23T08:12:58.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAJDJxpmbUwNhAA2tMEaf+OxLKl1WUGaNfMQw== --osd-uuid b963bfde-d129-4e47-aff2-f14d7bf4e54d 2026-02-23T08:12:58.152 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:58.151+0000 7fc0c2e02a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:58.153 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:58.151+0000 7fc0c2e02a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:58.154 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:58.151+0000 7fc0c2e02a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:58.155 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:58.151+0000 7fc0c2e02a40 -1 bdev(0x55d136b46400 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:12:58.155 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:58.151+0000 7fc0c2e02a40 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-23T08:12:58.794 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-23T08:12:58.794 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:12:58.795 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:12:58.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:12:58.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:12:59.081 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:12:59.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:12:59.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:12:59.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:12:59.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:12:59.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:12:59.103 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:59.099+0000 7f749a5b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:59.105 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:59.103+0000 7f749a5b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:59.106 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:59.103+0000 7f749a5b6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:12:59.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:12:59.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:12:59.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:12:59.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:12:59.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:12:59.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:12:59.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:12:59.301 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:12:59.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:12:59.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:12:59.437 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:12:59.435+0000 7f749a5b6a40 -1 Falling back to public interface 2026-02-23T08:12:59.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:00.149 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:00.147+0000 7f749a5b6a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:13:00.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:00.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:00.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:00.512 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:00.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:00.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:13:00.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:01.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:01.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:01.729 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:01.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:01.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:01.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:13:01.944 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3223940301,v1:127.0.0.1:6819/3223940301] [v2:127.0.0.1:6820/3223940301,v1:127.0.0.1:6821/3223940301] exists,up b963bfde-d129-4e47-aff2-f14d7bf4e54d 2026-02-23T08:13:01.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:01.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:01.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:01.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-23T08:13:01.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-23T08:13:02.142 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' does not exist 2026-02-23T08:13:02.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-23T08:13:02.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:13:02.388 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:13:02.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:13:03.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-23T08:13:03.681 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-23T08:13:03.681 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:13:03.681 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:13:03.681 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:13:03.681 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:13:03.681 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:13:03.681 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:13:03.682 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:13:03.682 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:13:03.682 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:13:03.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:13:03.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:13:03.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:13:03.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:13:03.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:13:03.778 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:13:03.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:13:03.984 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:13:03.984 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:13:03.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:13:03.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:13:03.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:13:04.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-23T08:13:04.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-23T08:13:04.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485' 2026-02-23T08:13:04.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:13:04.066 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:13:04.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-23T08:13:04.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-23T08:13:04.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-23T08:13:04.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:13:04.141 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:13:04.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-23T08:13:04.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-23T08:13:04.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542148' 2026-02-23T08:13:04.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:13:04.221 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836485 2026-02-23T08:13:04.221 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:13:04.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:13:04.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-23T08:13:04.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:13:04.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836485 2026-02-23T08:13:04.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-23T08:13:04.224 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 21474836485 2026-02-23T08:13:04.224 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:13:04.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836485 2026-02-23T08:13:04.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:13:05.440 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:13:05.441 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:13:05.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-23T08:13:05.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:13:05.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-23T08:13:05.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:13:05.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:13:05.657 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-23T08:13:05.657 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:13:05.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-23T08:13:05.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-23T08:13:05.659 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 42949672964 2026-02-23T08:13:05.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:13:05.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-23T08:13:05.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:13:05.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542148 2026-02-23T08:13:05.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:13:05.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:13:05.875 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-23T08:13:05.875 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:13:05.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542148 2026-02-23T08:13:05.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-23T08:13:05.877 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 60129542148 2026-02-23T08:13:05.877 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:13:06.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-23T08:13:06.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:13:06.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:13:06.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:13:06.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-23T08:13:06.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:13:06.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:13:06.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:13:06.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:13:06.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:13:06.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:13:06.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:13:06.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-23T08:13:06.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:13:06.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:13:06.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:13:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-23T08:13:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:13:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:13:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-23T08:13:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-23T08:13:07.150 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-23T08:13:07.150 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:07.150 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:07.151 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:07.151 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:07.151 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:07.151 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:07.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-23T08:13:07.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:07.201 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:07.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:07.210 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.027+0000 7f5cc996d640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.210 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.139+0000 7f5cc996d640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.210 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.139+0000 7f5cc996d640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-23T08:13:07.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-23T08:13:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:07.504 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:07.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:07.512 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.375+0000 7f5cc996d640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.513 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.439+0000 7f5cc996d640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.513 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.439+0000 7f5cc996d640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-23T08:13:07.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-23T08:13:07.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-23T08:13:07.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:07.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:07.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:07.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:07.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:07.788 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:07.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-23T08:13:07.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:07.840 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:07.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:07.849 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.663+0000 7f5cc996d640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.849 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.771+0000 7f5cc996d640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.849 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:07.771+0000 7f5cc996d640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-23T08:13:07.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-23T08:13:07.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-23T08:13:08.129 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-23T08:13:08.129 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:08.129 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:08.129 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:08.129 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:08.129 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:08.130 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:08.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-23T08:13:08.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:08.181 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:08.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:08.189 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.007+0000 7f5cc996d640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.189 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.115+0000 7f5cc996d640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.189 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.115+0000 7f5cc996d640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-23T08:13:08.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-23T08:13:08.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-23T08:13:08.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:08.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:08.471 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:08.471 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:08.471 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:08.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:08.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-23T08:13:08.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:08.523 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:08.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:08.531 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.347+0000 7f5cc996d640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.532 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.459+0000 7f5cc996d640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.532 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.459+0000 7f5cc996d640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-23T08:13:08.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-23T08:13:08.773 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-23T08:13:08.773 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:08.773 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:08.773 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:08.773 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:08.773 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:08.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:08.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-23T08:13:08.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:08.823 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:08.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:08.832 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.691+0000 7f5cc996d640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.832 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.759+0000 7f5cc996d640 0 mon.a@0(leader).osd e28 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.832 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:08.759+0000 7f5cc996d640 0 mon.a@0(leader).osd e28 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-23T08:13:08.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:08.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:08.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:08.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:08.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:08.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:08.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:08.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:08.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:08.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:08.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:08.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:08.945 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:08.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:08.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:08.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:08.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:08.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:08.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:08.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:08.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:08.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:08.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:08.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:08.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:08.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:08.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:08.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:08.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:08.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:08.963 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:08.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:08.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:08.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:08.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:08.966 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:08.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:08.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:08.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:08.968 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:08.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:08.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:08.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:08.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:08.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:08.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:08.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:08.972 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:08.972 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:08.972 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:08.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:08.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:08.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:08.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:08.975 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:08.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:08.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:08.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:08.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-23T08:13:08.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:08.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:08.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:08.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:08.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:08.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-23T08:13:08.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:08.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:08.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:08.998 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:08.998 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:08.998 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:08.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:08.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-23T08:13:09.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:09.028 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:09.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:09.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:09.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:09.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:09.030 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:09.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:09.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:09.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:09.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:09.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:09.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:09.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:09.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:09.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:09.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:09.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:09.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:09.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:09.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:09.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:09.258 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:09.258 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:09.258 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:09.258 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:09.258 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:09.258 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:09.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:09.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-23T08:13:09.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:09.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:09.276 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:09.276 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:09.277 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:09.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:09.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:09.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:09.327 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:09.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:09.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:09.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c82ab480-2b55-4d69-b10a-4b36369607e6 2026-02-23T08:13:09.337 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 c82ab480-2b55-4d69-b10a-4b36369607e6 2026-02-23T08:13:09.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c82ab480-2b55-4d69-b10a-4b36369607e6' 2026-02-23T08:13:09.337 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:09.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAVDJxpi0fAFBAAbF9N/lGqFvGAmCgY4RL7QQ== 2026-02-23T08:13:09.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAVDJxpi0fAFBAAbF9N/lGqFvGAmCgY4RL7QQ=="}' 2026-02-23T08:13:09.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c82ab480-2b55-4d69-b10a-4b36369607e6 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:09.459 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:09.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:09.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQAVDJxpi0fAFBAAbF9N/lGqFvGAmCgY4RL7QQ== --osd-uuid c82ab480-2b55-4d69-b10a-4b36369607e6 2026-02-23T08:13:09.482 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:09.479+0000 7fa926ac3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:09.483 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:09.479+0000 7fa926ac3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:09.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:09.483+0000 7fa926ac3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:09.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:09.483+0000 7fa926ac3a40 -1 bdev(0x55684a604400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:09.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:09.483+0000 7fa926ac3a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:10.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:10.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:10.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:10.254 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:10.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:10.378 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:10.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:10.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-23T08:13:10.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:10.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:10.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:10.400 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:10.395+0000 7febb2589a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:10.402 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:10.399+0000 7febb2589a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:10.403 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:10.399+0000 7febb2589a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:10.489 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:10.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:10.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:10.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:10.829 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:10.827+0000 7febb2589a40 -1 Falling back to public interface 2026-02-23T08:13:11.573 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:11.571+0000 7febb2589a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:11.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:11.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:11.599 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:11.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:11.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:11.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:11.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:12.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:12.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:12.810 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:12.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:12.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:12.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:13.025 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2810001966,v1:127.0.0.1:6801/2810001966] [v2:127.0.0.1:6802/2810001966,v1:127.0.0.1:6803/2810001966] exists,up c82ab480-2b55-4d69-b10a-4b36369607e6 2026-02-23T08:13:13.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:13.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:13.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:13.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:13.094 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:13.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:13.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:13.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:13.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:13.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:13.212 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:13.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:13.213 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:13.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:13.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:13.215 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:13.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:13.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:13.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:13.217 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:13.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:13.218 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:13.218 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:13.223 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:13.223 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.223 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:13.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:13.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:13.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:13.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:13.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:13.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:13.228 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:13.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:13.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:13.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:13.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:13.231 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:13.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:13.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:13.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:13.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:13.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:13.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:13.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:13.236 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:13.236 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.236 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.236 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:13.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:13.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:13.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:13.239 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:13.239 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.239 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:13.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:13.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:13.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:13.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-23T08:13:13.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:13.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:13.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:13.263 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:13.263 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.263 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:13.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-23T08:13:13.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:13.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:13.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:13.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:13.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:13.292 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:13.292 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:13.293 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:13.293 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:13.293 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:13.293 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.293 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.293 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:13.293 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:13.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:13.351 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:13.352 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.352 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.352 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:13.353 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:13.353 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:13.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:13.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:13.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:13.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:13.522 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:13.522 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:13.522 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:13.522 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:13.522 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.522 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.522 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:13.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:13.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:13.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:13.593 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:13.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-23T08:13:13.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:13.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:13.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:13.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:13.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-23T08:13:13.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:13.603 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:13.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8dc1a822-091a-448c-9bff-68c24ddbdbf9 2026-02-23T08:13:13.604 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 8dc1a822-091a-448c-9bff-68c24ddbdbf9 2026-02-23T08:13:13.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8dc1a822-091a-448c-9bff-68c24ddbdbf9' 2026-02-23T08:13:13.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:13.614 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAZDJxprXmXJBAADeF8JiVRxlnWLCxMqp6rxg== 2026-02-23T08:13:13.614 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAZDJxprXmXJBAADeF8JiVRxlnWLCxMqp6rxg=="}' 2026-02-23T08:13:13.614 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8dc1a822-091a-448c-9bff-68c24ddbdbf9 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:13.722 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:13.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:13.731 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQAZDJxprXmXJBAADeF8JiVRxlnWLCxMqp6rxg== --osd-uuid 8dc1a822-091a-448c-9bff-68c24ddbdbf9 2026-02-23T08:13:13.746 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:13.743+0000 7f395f042a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:13.747 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:13.743+0000 7f395f042a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:13.748 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:13.747+0000 7f395f042a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:13.748 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:13.747+0000 7f395f042a40 -1 bdev(0x55e32af1e400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:13.748 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:13.747+0000 7f395f042a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:14.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:14.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:14.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:14.544 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:14.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:14.666 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:14.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:14.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-23T08:13:14.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:14.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:14.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:14.689 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:14.687+0000 7f8e0357ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:14.691 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:14.687+0000 7f8e0357ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:14.692 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:14.691+0000 7f8e0357ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:14.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:14.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:14.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:14.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:14.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:14.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:14.779 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:14.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:14.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:14.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:14.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:15.069 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:15.067+0000 7f8e0357ea40 -1 Falling back to public interface 2026-02-23T08:13:15.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:15.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:15.889 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:15.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:15.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:15.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:16.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:16.149 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:16.147+0000 7f8e0357ea40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:17.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:17.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:17.102 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:17.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:17.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:17.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:17.315 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1668701577,v1:127.0.0.1:6801/1668701577] [v2:127.0.0.1:6802/1668701577,v1:127.0.0.1:6803/1668701577] exists,up 8dc1a822-091a-448c-9bff-68c24ddbdbf9 2026-02-23T08:13:17.316 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:17.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:17.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:17.381 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:17.387 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:17.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:17.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:17.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:17.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:17.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:17.391 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:17.392 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:17.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:17.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:17.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:17.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:17.499 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:17.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:17.500 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:17.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:17.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:17.501 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:17.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:17.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:17.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:17.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:17.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:17.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:17.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:17.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:17.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:17.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:17.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:17.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:17.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:17.512 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:17.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:17.513 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:17.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:17.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:17.515 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:17.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:17.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:17.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:17.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:17.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:17.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:17.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:17.520 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:17.520 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.520 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:17.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:17.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:17.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:17.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:17.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:17.524 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:17.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:17.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:17.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-23T08:13:17.547 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:17.547 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:17.547 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:17.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:17.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.547 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:17.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-23T08:13:17.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:17.574 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:17.575 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:17.575 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.576 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.576 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:17.576 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:17.576 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:17.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:17.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:17.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:17.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:17.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:17.633 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:17.633 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:17.634 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:17.634 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:17.635 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.635 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.635 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:17.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:17.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:17.690 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:17.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:17.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:17.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:17.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:17.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:17.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:17.805 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:17.805 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:17.805 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:17.805 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:17.805 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.805 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.805 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:17.806 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:17.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:17.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:17.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:17.876 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:17.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-23T08:13:17.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:17.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:17.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:17.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:17.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:17.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:17.887 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:17.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:17.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:17.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:17.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-23T08:13:17.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:17.889 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:17.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=53c2a64e-e6af-46ab-8d57-3a17b564f6fe 2026-02-23T08:13:17.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 53c2a64e-e6af-46ab-8d57-3a17b564f6fe' 2026-02-23T08:13:17.890 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 53c2a64e-e6af-46ab-8d57-3a17b564f6fe 2026-02-23T08:13:17.890 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:17.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAdDJxpSmWANRAAqIxxjR23Cg0ejYapg7rcYA== 2026-02-23T08:13:17.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAdDJxpSmWANRAAqIxxjR23Cg0ejYapg7rcYA=="}' 2026-02-23T08:13:17.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 53c2a64e-e6af-46ab-8d57-3a17b564f6fe -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:18.008 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:18.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:18.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQAdDJxpSmWANRAAqIxxjR23Cg0ejYapg7rcYA== --osd-uuid 53c2a64e-e6af-46ab-8d57-3a17b564f6fe 2026-02-23T08:13:18.032 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.031+0000 7fe5aee70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:18.034 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.031+0000 7fe5aee70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:18.035 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.031+0000 7fe5aee70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:18.035 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.031+0000 7fe5aee70a40 -1 bdev(0x55cf0bf32400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:18.035 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.031+0000 7fe5aee70a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:18.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:18.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:18.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:18.780 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:18.781 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:18.906 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:18.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:18.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-23T08:13:18.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:18.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:18.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:18.928 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.927+0000 7f76d431da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:18.930 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.927+0000 7f76d431da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:18.931 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:18.927+0000 7f76d431da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:19.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:19.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:19.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:19.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:19.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:19.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:19.018 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:19.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:19.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:19.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:19.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:19.353 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:19.351+0000 7f76d431da40 -1 Falling back to public interface 2026-02-23T08:13:20.061 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:20.059+0000 7f76d431da40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:20.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:20.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:20.128 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:20.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:20.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:20.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:20.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:21.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:21.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:21.240 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:21.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:21.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:21.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:21.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:22.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:22.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:22.450 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-23T08:13:22.450 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:13:22.450 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:22.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:22.666 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4084958810,v1:127.0.0.1:6801/4084958810] [v2:127.0.0.1:6802/4084958810,v1:127.0.0.1:6803/4084958810] exists,up 53c2a64e-e6af-46ab-8d57-3a17b564f6fe 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:22.668 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:22.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:22.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:22.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:22.725 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:22.731 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:22.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:22.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:22.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:22.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:22.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:22.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:22.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:22.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:22.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:22.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:22.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:22.845 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:22.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:22.846 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:22.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:22.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:22.847 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:22.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:22.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:22.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:22.849 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:22.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:22.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:22.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:22.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:22.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:22.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:22.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:22.858 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:22.859 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:22.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:22.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:22.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:22.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:22.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:22.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:22.862 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:22.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:22.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:22.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:22.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:22.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:22.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:22.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:22.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:22.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:22.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:22.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:22.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:22.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:22.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:22.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:22.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:22.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:22.871 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:22.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:22.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:22.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:22.874 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:22.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:22.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:22.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:22.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-23T08:13:22.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:22.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:22.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:22.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:22.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:22.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-23T08:13:22.895 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:22.895 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:22.896 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:22.896 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:22.896 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:22.896 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:22.896 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:22.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-23T08:13:22.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:22.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:22.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:22.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:22.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:22.924 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:22.924 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:22.924 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:22.924 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:22.924 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:22.924 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:22.925 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:22.925 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:22.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:22.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:22.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:22.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:22.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:22.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:22.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:22.985 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:23.153 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:23.153 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:23.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:23.154 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:23.154 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:23.154 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:23.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:23.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:23.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:23.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:23.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:23.228 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:23.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-23T08:13:23.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:23.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:23.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:23.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-23T08:13:23.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:23.242 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:23.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bb784e3e-0353-4fd1-bded-a845442051b2 2026-02-23T08:13:23.242 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 bb784e3e-0353-4fd1-bded-a845442051b2 2026-02-23T08:13:23.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bb784e3e-0353-4fd1-bded-a845442051b2' 2026-02-23T08:13:23.243 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:23.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAjDJxpkc3XDhAA4FFcoQGF/XQS3lNC+8K68g== 2026-02-23T08:13:23.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAjDJxpkc3XDhAA4FFcoQGF/XQS3lNC+8K68g=="}' 2026-02-23T08:13:23.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bb784e3e-0353-4fd1-bded-a845442051b2 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:23.360 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:23.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:23.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQAjDJxpkc3XDhAA4FFcoQGF/XQS3lNC+8K68g== --osd-uuid bb784e3e-0353-4fd1-bded-a845442051b2 2026-02-23T08:13:23.384 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:23.383+0000 7f89c8ce6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:23.386 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:23.383+0000 7f89c8ce6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:23.387 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:23.383+0000 7f89c8ce6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:23.387 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:23.383+0000 7f89c8ce6a40 -1 bdev(0x561a296bc400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:23.388 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:23.383+0000 7f89c8ce6a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:24.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:24.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:24.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:24.011 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:24.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:24.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:24.122 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:24.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-23T08:13:24.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:24.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:24.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:24.135 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:24.131+0000 7f9338078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:24.137 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:24.135+0000 7f9338078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:24.138 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:24.135+0000 7f9338078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:24.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:24.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:24.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:24.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:24.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:24.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:24.234 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:24.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:24.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:24.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:24.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:24.581 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:24.579+0000 7f9338078a40 -1 Falling back to public interface 2026-02-23T08:13:25.227 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:25.223+0000 7f9338078a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:25.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:25.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:25.341 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:25.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:25.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:25.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:25.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:26.486 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:26.701 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2963047124,v1:127.0.0.1:6801/2963047124] [v2:127.0.0.1:6802/2963047124,v1:127.0.0.1:6803/2963047124] exists,up bb784e3e-0353-4fd1-bded-a845442051b2 2026-02-23T08:13:26.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:26.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-23T08:13:26.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:26.767 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:26.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:26.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:26.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:26.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:26.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:26.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:26.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:26.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:26.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:26.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:26.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:26.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:26.885 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:26.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:26.886 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:26.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:26.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:26.888 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:26.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:26.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:26.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:26.889 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:26.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:26.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:26.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:26.896 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:26.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:26.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:26.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:26.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:26.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:26.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-23T08:13:26.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:26.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:26.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:26.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:26.901 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:26.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:26.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:26.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:26.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:26.904 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:26.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:26.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:26.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:26.906 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:26.909 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:26.909 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:26.910 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:26.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:26.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:26.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:26.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:26.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:26.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:26.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:26.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:26.914 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:26.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:26.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:26.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:26.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-23T08:13:26.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:26.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:26.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:26.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:26.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:26.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-23T08:13:26.937 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:26.937 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:26.937 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:26.937 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:26.937 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:26.937 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:26.938 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:26.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:13:26.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:26.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:26.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:26.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:26.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:26.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:26.968 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:26.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:27.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:27.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:27.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:27.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:27.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:27.026 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:27.026 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:27.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:27.027 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:27.028 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:27.028 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:27.028 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:27.028 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:27.028 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:27.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-23T08:13:27.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:27.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:27.089 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:27.089 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:27.089 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:27.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:27.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-23T08:13:27.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:27.143 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:27.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:27.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:27.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:27.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:27.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:27.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:27.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:27.274 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:27.274 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:27.274 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:27.274 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:27.274 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:27.275 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:27.275 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:27.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:27.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:27.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-23T08:13:27.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:27.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:27.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:27.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:27.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:27.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:27.296 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:27.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:27.298 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:27.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0d18b805-b692-4c00-885d-80d09658b739 2026-02-23T08:13:27.299 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 0d18b805-b692-4c00-885d-80d09658b739 2026-02-23T08:13:27.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0d18b805-b692-4c00-885d-80d09658b739' 2026-02-23T08:13:27.300 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:27.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAnDJxpA2QqEhAAlpUfn+PYD5saQl9YVnUevg== 2026-02-23T08:13:27.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAnDJxpA2QqEhAAlpUfn+PYD5saQl9YVnUevg=="}' 2026-02-23T08:13:27.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0d18b805-b692-4c00-885d-80d09658b739 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:27.417 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:27.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAnDJxpA2QqEhAAlpUfn+PYD5saQl9YVnUevg== --osd-uuid 0d18b805-b692-4c00-885d-80d09658b739 2026-02-23T08:13:27.440 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:27.439+0000 7f1d8887ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:27.442 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:27.439+0000 7f1d8887ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:27.443 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:27.439+0000 7f1d8887ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:27.444 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:27.439+0000 7f1d8887ba40 -1 bdev(0x56135f000400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:27.444 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:27.443+0000 7f1d8887ba40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:28.147 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:28.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:28.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:28.149 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:28.149 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:28.261 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:28.261 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:28.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:13:28.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:28.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:28.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:28.276 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:28.275+0000 7f43f2207a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:28.277 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:28.275+0000 7f43f2207a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:28.278 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:28.275+0000 7f43f2207a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:28.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:28.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:28.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:28.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:28.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:28.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:28.378 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:28.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:28.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:28.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:28.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:28.658 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:28.655+0000 7f43f2207a40 -1 Falling back to public interface 2026-02-23T08:13:29.236 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:29.235+0000 7f43f2207a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:29.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:29.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:29.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:29.490 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:29.491 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:29.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:29.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:30.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:30.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:30.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:30.632 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:30.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:30.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:30.855 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2012176934,v1:127.0.0.1:6801/2012176934] [v2:127.0.0.1:6802/2012176934,v1:127.0.0.1:6803/2012176934] exists,up 0d18b805-b692-4c00-885d-80d09658b739 2026-02-23T08:13:30.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:30.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:30.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:30.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-23T08:13:30.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:30.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:30.856 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:30.856 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:30.856 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:30.856 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-23T08:13:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:30.920 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:30.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:30.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:30.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-23T08:13:30.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:30.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:30.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:30.930 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:30.930 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:30.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:30.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:30.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:31.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:31.039 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:31.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:31.040 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:31.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:31.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:31.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:31.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:31.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:31.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:31.043 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:31.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:31.045 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:31.045 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:31.049 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:31.049 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.049 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:31.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:31.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:31.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:31.054 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:31.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:31.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:31.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:31.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:31.056 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:31.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:31.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:31.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:31.058 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:31.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:31.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:31.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:31.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:31.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:31.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:31.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:31.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:31.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:31.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:31.066 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:31.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-23T08:13:31.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:31.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:31.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:31.088 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:31.088 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.088 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:31.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-23T08:13:31.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:31.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:31.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:31.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:31.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:31.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:31.117 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:31.117 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:31.117 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:31.117 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:31.117 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.117 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.117 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:31.118 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:31.118 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:31.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:31.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:31.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:31.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:31.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:31.175 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:31.175 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:31.175 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:31.176 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:31.176 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.176 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.176 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:31.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:31.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:31.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:31.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:31.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:31.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:31.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:31.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:31.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:31.342 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:31.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:31.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:31.343 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:31.343 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.343 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.344 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:31.344 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:31.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:31.361 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-23T08:13:31.361 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:31.361 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:31.361 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:31.361 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.362 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:31.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:31.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:31.415 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-23T08:13:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:31.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-23T08:13:31.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:31.429 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:31.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=56c3a226-f2e4-457f-bd04-d3fe585b630e 2026-02-23T08:13:31.429 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 56c3a226-f2e4-457f-bd04-d3fe585b630e 2026-02-23T08:13:31.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 56c3a226-f2e4-457f-bd04-d3fe585b630e' 2026-02-23T08:13:31.430 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:31.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQArDJxpZLL1GRAAncG0pcPnlZ61zShKxLYCGg== 2026-02-23T08:13:31.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQArDJxpZLL1GRAAncG0pcPnlZ61zShKxLYCGg=="}' 2026-02-23T08:13:31.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 56c3a226-f2e4-457f-bd04-d3fe585b630e -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:31.549 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:31.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:31.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQArDJxpZLL1GRAAncG0pcPnlZ61zShKxLYCGg== --osd-uuid 56c3a226-f2e4-457f-bd04-d3fe585b630e 2026-02-23T08:13:31.574 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:31.571+0000 7f5389a18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:31.575 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:31.571+0000 7f5389a18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:31.576 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:31.571+0000 7f5389a18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:31.576 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:31.575+0000 7f5389a18a40 -1 bdev(0x55d8ad060400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:31.576 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:31.575+0000 7f5389a18a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:32.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:32.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:32.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:32.324 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:32.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:32.438 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:32.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:32.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-23T08:13:32.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:32.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:32.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:32.461 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:32.459+0000 7f6327276a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:32.463 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:32.459+0000 7f6327276a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:32.464 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:32.463+0000 7f6327276a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:32.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:32.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:32.938 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:32.935+0000 7f6327276a40 -1 Falling back to public interface 2026-02-23T08:13:33.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:33.659 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:33.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:33.660 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:33.660 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:33.660 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:33.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:33.882 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:33.879+0000 7f6327276a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:34.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:34.866 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:34.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:34.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:34.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:34.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:35.074 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2188678069,v1:127.0.0.1:6801/2188678069] [v2:127.0.0.1:6802/2188678069,v1:127.0.0.1:6803/2188678069] exists,up 56c3a226-f2e4-457f-bd04-d3fe585b630e 2026-02-23T08:13:35.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:35.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:35.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:35.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-23T08:13:35.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:35.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:35.075 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:35.075 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.075 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:35.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:35.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:35.131 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:35.137 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:31.099+0000 7fd66187bec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-23T08:13:35.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:35.138 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:32.935+0000 7f6327276a40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:35.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:35.248 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:35.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:35.250 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:35.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:35.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:35.251 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:35.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:35.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:35.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:35.253 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:35.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:35.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:35.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:35.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:35.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:35.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:35.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:35.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:35.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:35.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:35.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:35.264 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:35.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:35.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:35.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:35.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:35.266 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:35.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:35.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:35.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:35.268 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:35.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:35.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:35.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:35.271 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:35.271 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.271 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:35.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:35.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:35.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:35.273 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:35.274 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.274 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.274 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:35.275 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:35.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:35.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-23T08:13:35.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:35.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:35.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:35.300 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:35.300 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.300 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.300 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:35.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-23T08:13:35.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:35.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:35.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:35.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:35.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:35.331 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:35.331 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:35.331 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:35.331 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:35.331 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:35.331 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.331 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.332 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:35.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:35.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:35.384 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:35.384 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.385 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.386 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:35.386 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:35.386 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:35.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:35.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:35.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:35.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:35.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:35.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:35.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:35.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:35.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:35.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:35.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:35.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.547 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:35.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:35.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:35.566 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:35.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:35.615 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:35.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-23T08:13:35.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:35.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:35.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:35.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:35.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:35.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=29684be2-e141-44b0-ab56-4228786fe968 2026-02-23T08:13:35.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 29684be2-e141-44b0-ab56-4228786fe968' 2026-02-23T08:13:35.629 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 29684be2-e141-44b0-ab56-4228786fe968 2026-02-23T08:13:35.630 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:35.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAvDJxpCK3wJRAAnhLt/H7TI0y3oxZ9DaUrFw== 2026-02-23T08:13:35.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAvDJxpCK3wJRAAnhLt/H7TI0y3oxZ9DaUrFw=="}' 2026-02-23T08:13:35.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 29684be2-e141-44b0-ab56-4228786fe968 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:35.751 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:35.759 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:35.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQAvDJxpCK3wJRAAnhLt/H7TI0y3oxZ9DaUrFw== --osd-uuid 29684be2-e141-44b0-ab56-4228786fe968 2026-02-23T08:13:35.776 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:35.775+0000 7f34d6539a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:35.778 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:35.775+0000 7f34d6539a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:35.779 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:35.775+0000 7f34d6539a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:35.779 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:35.775+0000 7f34d6539a40 -1 bdev(0x555f0c26e400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:35.779 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:35.775+0000 7f34d6539a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:36.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:36.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:36.503 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:36.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:36.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:36.623 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:36.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:36.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-23T08:13:36.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:36.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:36.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:36.647 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:36.643+0000 7ff4fe177a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:36.649 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:36.647+0000 7ff4fe177a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:36.650 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:36.647+0000 7ff4fe177a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:36.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:36.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:37.026 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:37.023+0000 7ff4fe177a40 -1 Falling back to public interface 2026-02-23T08:13:37.649 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:37.647+0000 7ff4fe177a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:37.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:37.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:37.840 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:37.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:37.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:37.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:37.955 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:38.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:38.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:38.957 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:38.957 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:38.957 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:38.957 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:39.169 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3237374939,v1:127.0.0.1:6801/3237374939] [v2:127.0.0.1:6802/3237374939,v1:127.0.0.1:6803/3237374939] exists,up 29684be2-e141-44b0-ab56-4228786fe968 2026-02-23T08:13:39.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:39.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:39.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:39.169 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-23T08:13:39.169 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:39.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:39.170 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:39.170 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.171 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.171 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:39.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:39.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:39.225 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:39.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:39.232 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:35.311+0000 7ff616297ec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-23T08:13:39.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:39.233 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:37.023+0000 7ff4fe177a40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-23T08:13:39.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:39.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:39.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:39.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:39.343 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:39.344 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:39.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:39.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:39.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:39.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:39.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:39.347 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:39.347 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:39.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:39.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:39.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:39.354 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:39.354 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.354 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:39.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:39.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:39.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-23T08:13:39.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:39.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:39.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:39.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:39.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:39.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:39.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:39.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:39.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:39.361 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:39.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:39.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:39.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:39.363 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:39.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:39.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:39.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:39.366 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:39.366 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.366 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:39.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:39.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:39.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:39.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:39.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:39.371 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:39.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-23T08:13:39.393 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:39.393 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:39.393 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:39.394 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:39.394 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.394 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.394 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:39.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-23T08:13:39.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:39.424 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:39.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:39.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:39.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:39.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:39.425 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:39.425 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:39.425 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:39.426 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:39.426 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.426 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.427 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:39.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:39.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:39.482 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:39.482 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:39.482 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:39.483 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:39.483 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:39.483 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:39.483 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:39.483 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:39.485 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:39.485 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.485 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.485 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:39.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:39.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:39.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:39.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:39.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:39.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:39.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:39.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:39.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:39.650 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:39.650 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:39.650 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:39.650 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:39.650 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.650 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.650 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:39.651 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:39.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:39.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-23T08:13:39.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:39.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:39.670 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:39.670 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.670 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:39.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:39.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:39.721 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:39.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:39.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-23T08:13:39.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:39.731 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:39.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3271acd0-1329-4fcd-ad18-3557b5ab6f34 2026-02-23T08:13:39.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3271acd0-1329-4fcd-ad18-3557b5ab6f34' 2026-02-23T08:13:39.732 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 3271acd0-1329-4fcd-ad18-3557b5ab6f34 2026-02-23T08:13:39.732 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:39.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAzDJxpTuVFLBAAr0VflTNxrIdkLltcUrpK8Q== 2026-02-23T08:13:39.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAzDJxpTuVFLBAAr0VflTNxrIdkLltcUrpK8Q=="}' 2026-02-23T08:13:39.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3271acd0-1329-4fcd-ad18-3557b5ab6f34 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:39.857 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:39.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:39.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQAzDJxpTuVFLBAAr0VflTNxrIdkLltcUrpK8Q== --osd-uuid 3271acd0-1329-4fcd-ad18-3557b5ab6f34 2026-02-23T08:13:39.882 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:39.879+0000 7f58a7198a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:39.883 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:39.879+0000 7f58a7198a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:39.884 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:39.883+0000 7f58a7198a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:39.884 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:39.883+0000 7f58a7198a40 -1 bdev(0x55fabb742400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:39.885 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:39.883+0000 7f58a7198a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:40.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:40.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:40.629 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:40.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:40.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:40.748 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:40.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:40.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-23T08:13:40.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:40.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:40.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:40.773 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:40.771+0000 7fe3e4388a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:40.775 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:40.771+0000 7fe3e4388a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:40.776 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:40.775+0000 7fe3e4388a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:40.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:40.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:40.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:40.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:40.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:40.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:40.858 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:40.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:40.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:40.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:40.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:41.177 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:41.175+0000 7fe3e4388a40 -1 Falling back to public interface 2026-02-23T08:13:41.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:41.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:41.966 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:41.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:41.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:41.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:42.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:42.161 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:42.159+0000 7fe3e4388a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:43.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:43.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:43.074 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:43.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:43.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:43.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:43.288 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3648666656,v1:127.0.0.1:6801/3648666656] [v2:127.0.0.1:6802/3648666656,v1:127.0.0.1:6803/3648666656] exists,up 3271acd0-1329-4fcd-ad18-3557b5ab6f34 2026-02-23T08:13:43.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:43.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:43.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:43.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-23T08:13:43.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:43.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:43.291 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:43.291 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.291 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.291 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:43.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:43.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:43.352 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:43.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:43.359 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:39.407+0000 7fe0d9eb2ec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-23T08:13:43.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:43.361 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:41.175+0000 7fe3e4388a40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-23T08:13:43.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:43.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:43.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:43.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:43.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:43.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:43.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:43.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:43.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:43.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:43.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:43.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:43.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:43.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:43.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:43.473 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:43.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:43.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:43.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:43.475 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:43.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:43.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:43.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:43.480 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:43.480 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.480 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:43.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:43.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:43.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-23T08:13:43.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:43.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:43.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:43.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:43.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:43.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:43.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:43.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:43.487 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:43.488 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:43.488 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:43.488 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:43.489 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:43.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:43.491 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:43.491 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:43.492 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:43.492 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.492 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:43.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:43.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:43.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:43.495 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:43.495 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.495 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:43.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:43.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:43.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:43.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-23T08:13:43.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:43.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:43.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:43.519 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:43.519 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.519 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.520 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:43.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-23T08:13:43.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:43.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:43.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:43.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:43.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:43.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:43.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:43.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:43.607 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.608 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.609 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:43.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:43.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:43.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:43.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:43.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:43.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:43.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:43.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:43.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:43.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:43.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:43.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:43.776 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:43.777 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.777 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.777 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:43.777 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:43.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:43.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-23T08:13:43.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:43.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:43.796 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:43.796 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.796 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:43.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:43.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:43.846 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:43.853 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-23T08:13:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:43.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:43.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f1976674-3f9b-4c0f-9b8b-45293752687f 2026-02-23T08:13:43.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f1976674-3f9b-4c0f-9b8b-45293752687f' 2026-02-23T08:13:43.856 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 f1976674-3f9b-4c0f-9b8b-45293752687f 2026-02-23T08:13:43.857 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:43.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA3DJxpceqwMxAA6U69NVJLtCtRMZyJKp6p0w== 2026-02-23T08:13:43.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA3DJxpceqwMxAA6U69NVJLtCtRMZyJKp6p0w=="}' 2026-02-23T08:13:43.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f1976674-3f9b-4c0f-9b8b-45293752687f -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:43.979 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:43.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:43.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQA3DJxpceqwMxAA6U69NVJLtCtRMZyJKp6p0w== --osd-uuid f1976674-3f9b-4c0f-9b8b-45293752687f 2026-02-23T08:13:44.004 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.003+0000 7f36dd97da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:44.006 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.003+0000 7f36dd97da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:44.007 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.003+0000 7f36dd97da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:44.007 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.003+0000 7f36dd97da40 -1 bdev(0x55e3805fa400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:44.007 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.003+0000 7f36dd97da40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:44.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:44.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:44.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:44.658 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:44.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:44.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:44.774 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:44.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-23T08:13:44.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:44.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:44.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:44.788 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.787+0000 7fd4bd069a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:44.790 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.787+0000 7fd4bd069a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:44.791 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:44.787+0000 7fd4bd069a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:44.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:44.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:44.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:44.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:44.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:44.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:44.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:44.881 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:44.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:44.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:44.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:45.277 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:45.275+0000 7fd4bd069a40 -1 Falling back to public interface 2026-02-23T08:13:45.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:45.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:45.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:45.988 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:45.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:45.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:46.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:46.305 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:46.304+0000 7fd4bd069a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:47.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:47.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:47.118 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:47.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:47.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:47.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:47.328 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/253636730,v1:127.0.0.1:6801/253636730] [v2:127.0.0.1:6802/253636730,v1:127.0.0.1:6803/253636730] exists,up f1976674-3f9b-4c0f-9b8b-45293752687f 2026-02-23T08:13:47.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:47.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:47.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:47.329 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-23T08:13:47.329 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:47.329 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:47.330 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:47.330 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.330 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:47.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:47.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:47.396 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:47.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:47.403 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:43.535+0000 7f2d533a9ec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-23T08:13:47.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:47.405 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:45.275+0000 7fd4bd069a40 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-23T08:13:47.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:47.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:47.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:47.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:47.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:47.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:47.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:47.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:47.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:47.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:47.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:47.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:47.515 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:47.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:47.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:47.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:47.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:47.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:47.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:47.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:47.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:47.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:47.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:47.525 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:47.526 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.526 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-23T08:13:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:47.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:47.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:47.531 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:47.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:47.532 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:47.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:47.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:47.533 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:47.534 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:47.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:47.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:47.535 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:47.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:47.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:47.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:47.538 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:47.538 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.538 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:47.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:47.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:47.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:47.541 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:47.541 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:47.543 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:47.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:47.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-23T08:13:47.566 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:47.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:47.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:47.567 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:47.567 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.567 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:47.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-23T08:13:47.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:47.595 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:47.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:47.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:47.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:47.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:47.596 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:47.596 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:47.596 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:47.597 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:47.597 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.597 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.597 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:47.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:47.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:47.655 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.656 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.657 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:47.657 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:47.657 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:47.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:47.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:47.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:47.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:47.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:47.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:47.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:47.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:47.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:47.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:47.828 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:47.828 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.828 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:47.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:47.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:47.847 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-23T08:13:47.847 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:47.847 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:47.847 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:47.847 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.847 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.848 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:47.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:47.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:47.898 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:47.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-23T08:13:47.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:47.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:47.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:47.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:47.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:47.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:47.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=108cb64b-6fb4-4e5e-9a1d-593bcd3c9336 2026-02-23T08:13:47.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 108cb64b-6fb4-4e5e-9a1d-593bcd3c9336' 2026-02-23T08:13:47.912 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 108cb64b-6fb4-4e5e-9a1d-593bcd3c9336 2026-02-23T08:13:47.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:47.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA7DJxpJdXMNhAAyqZlnaFlQJK+vLrU6lAnJg== 2026-02-23T08:13:47.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA7DJxpJdXMNhAAyqZlnaFlQJK+vLrU6lAnJg=="}' 2026-02-23T08:13:47.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 108cb64b-6fb4-4e5e-9a1d-593bcd3c9336 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:48.028 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:48.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:48.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQA7DJxpJdXMNhAAyqZlnaFlQJK+vLrU6lAnJg== --osd-uuid 108cb64b-6fb4-4e5e-9a1d-593bcd3c9336 2026-02-23T08:13:48.052 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.052+0000 7f827c014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:48.054 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.052+0000 7f827c014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:48.055 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.052+0000 7f827c014a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:48.055 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.052+0000 7f827c014a40 -1 bdev(0x55b028d9c400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:48.055 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.052+0000 7f827c014a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:48.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:48.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:48.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:48.803 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:48.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:48.926 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:48.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:48.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-23T08:13:48.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:48.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:48.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:48.948 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.944+0000 7fc7aa4d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:48.950 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.944+0000 7fc7aa4d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:48.951 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:48.948+0000 7fc7aa4d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:49.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:49.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:49.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:49.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:49.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:49.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:49.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:49.040 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:49.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:49.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:49.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:49.385 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:49.380+0000 7fc7aa4d1a40 -1 Falling back to public interface 2026-02-23T08:13:50.112 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:50.108+0000 7fc7aa4d1a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:50.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:50.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:50.153 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:50.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:50.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:50.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:50.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:51.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:51.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:51.377 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:51.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:51.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:51.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:51.599 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4127997152,v1:127.0.0.1:6801/4127997152] [v2:127.0.0.1:6802/4127997152,v1:127.0.0.1:6803/4127997152] exists,up 108cb64b-6fb4-4e5e-9a1d-593bcd3c9336 2026-02-23T08:13:51.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:51.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:51.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:51.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-23T08:13:51.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:51.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:51.600 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:51.600 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:51.600 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:51.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:51.665 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:51.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:51.673 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:47.584+0000 7f83953e5ec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-23T08:13:51.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:51.674 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:49.380+0000 7fc7aa4d1a40 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-23T08:13:51.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:51.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:51.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:51.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:51.675 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:51.675 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:51.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:51.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:51.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:51.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:51.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:51.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:51.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:51.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:51.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:51.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:51.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:51.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:51.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:51.789 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:51.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:51.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:51.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:51.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:51.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:51.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:51.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:51.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:51.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:51.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:51.801 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:51.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:51.802 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:51.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:51.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:51.804 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:51.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:51.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:51.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:51.805 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:51.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:51.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:51.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:51.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:51.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:51.809 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:51.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:51.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:51.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:51.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-23T08:13:51.811 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:51.811 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:51.811 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:51.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11935 2026-02-23T08:13:51.813 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-23T08:13:51.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-23T08:13:51.835 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:51.835 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:51.836 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:51.836 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:51.836 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:51.836 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:51.836 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:51.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-23T08:13:51.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:51.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:51.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:51.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:51.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:51.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:51.869 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:51.869 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:51.869 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:51.870 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:51.870 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:51.870 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:51.870 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:51.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:51.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get fsid 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:51.927 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:51.928 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:51.928 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:51.928 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:51.928 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:51.928 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:51.928 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11935/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-23T08:13:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-23T08:13:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:52.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:52.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:52.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:52.098 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:52.098 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:52.098 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:52.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:52.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:52.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:52.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-23T08:13:52.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:52.117 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:52.117 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:52.117 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:52.117 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:52.117 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-mon.a.asok 2026-02-23T08:13:52.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:52.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-mon.a.asok log flush 2026-02-23T08:13:52.171 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:52.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-23T08:13:52.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:52.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:52.179 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-23T08:13:52.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-23T08:13:52.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eafb33f6-94f5-44bb-aa65-59ead5c760e3 2026-02-23T08:13:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 eafb33f6-94f5-44bb-aa65-59ead5c760e3' 2026-02-23T08:13:52.182 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 eafb33f6-94f5-44bb-aa65-59ead5c760e3 2026-02-23T08:13:52.182 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:52.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBADJxp+p6GCxAAlxhklNSy1dHxk2H7ksAQsw== 2026-02-23T08:13:52.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBADJxp+p6GCxAAlxhklNSy1dHxk2H7ksAQsw=="}' 2026-02-23T08:13:52.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eafb33f6-94f5-44bb-aa65-59ead5c760e3 -i td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:52.304 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:52.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-23T08:13:52.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQBADJxp+p6GCxAAlxhklNSy1dHxk2H7ksAQsw== --osd-uuid eafb33f6-94f5-44bb-aa65-59ead5c760e3 2026-02-23T08:13:52.328 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:52.324+0000 7fb36ac7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:52.330 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:52.324+0000 7fb36ac7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:52.330 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:52.324+0000 7fb36ac7aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:52.331 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:52.324+0000 7fb36ac7aa40 -1 bdev(0x55ba0b39e400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:52.331 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:52.328+0000 7fb36ac7aa40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-23T08:13:53.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-23T08:13:53.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:53.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:53.015 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:53.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:53.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:53.132 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:53.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62250c03-4b2a-4c00-98d2-edb8dfa7b828 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-23T08:13:53.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:53.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:53.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:53.154 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:53.148+0000 7f2b17f11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:53.156 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:53.152+0000 7f2b17f11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:53.157 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:53.152+0000 7f2b17f11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:53.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:53.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:53.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:53.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:53.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:53.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:53.244 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:53.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:53.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:53.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:53.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:53.517 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:53.512+0000 7f2b17f11a40 -1 Falling back to public interface 2026-02-23T08:13:54.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:54.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:54.358 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:54.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:54.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:54.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:54.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:54.528 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:54.524+0000 7f2b17f11a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:55.464 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:55.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:55.465 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:55.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:55.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:55.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:55.683 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1889741776,v1:127.0.0.1:6801/1889741776] [v2:127.0.0.1:6802/1889741776,v1:127.0.0.1:6803/1889741776] exists,up eafb33f6-94f5-44bb-aa65-59ead5c760e3 2026-02-23T08:13:55.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:13:55.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:13:55.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:13:55.684 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-23T08:13:55.685 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:13:55.685 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:13:55.685 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:55.685 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:55.685 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:55.685 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11935/ceph-osd.0.asok 2026-02-23T08:13:55.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-23T08:13:55.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11935/ceph-osd.0.asok log flush 2026-02-23T08:13:55.750 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:55.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-23T08:13:55.758 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:51.848+0000 7f1f90a59ec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-23T08:13:55.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-23T08:13:55.759 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:53.512+0000 7f2b17f11a40 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-23T08:13:55.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-23T08:13:55.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:55.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:55.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:55.760 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:55.760 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:55.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:55.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:55.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:55.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:55.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:55.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:55.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:55.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:55.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:55.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:55.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:55.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:55.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:55.874 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:55.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:55.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:55.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:55.879 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:55.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:55.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:55.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:55.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:55.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:55.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-23T08:13:55.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-23T08:13:55.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code-plugins 0 2026-02-23T08:13:55.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-23T08:13:55.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-23T08:13:55.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-23T08:13:55.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:55.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:55.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:55.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:55.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:55.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:55.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:55.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:55.885 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:55.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:55.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:55.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:55.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:55.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:55.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:55.888 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:55.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:55.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-23T08:13:55.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-23T08:13:55.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:55.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:55.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11935 2026-02-23T08:13:55.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11935 2026-02-23T08:13:55.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:55.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:55.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-23T08:13:55.895 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-23T08:13:55.895 DEBUG:teuthology.orchestra.run.trial186:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-23T08:13:55.943 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-23T08:13:55.943 DEBUG:teuthology.orchestra.run.trial186:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5adce3ff0fe1645dcbc38d059f948571dc3d69da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-23T08:13:55.996 INFO:tasks.workunit.client.0.trial186.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-23T08:13:56.000 INFO:tasks.workunit.client.0.trial186.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-23T08:13:56.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-23T08:13:56.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-23T08:13:56.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-23T08:13:56.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-23T08:13:56.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-23T08:13:56.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-23T08:13:56.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-23T08:13:56.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none ' 2026-02-23T08:13:56.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:13:56.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:13:56.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:13:56.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:13:56.006 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:13:56.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:13:56.006 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:13:56.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:13:56.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:13:56.007 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:13:56.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:56.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:13:56.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:13:56.008 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:13:56.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:13:56.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:13:56.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-23T08:13:56.012 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:13:56.012 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.012 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46012 2026-02-23T08:13:56.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:13:56.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:13:56.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-23T08:13:56.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:13:56.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.46012 2026-02-23T08:13:56.016 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:13:56.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-23T08:13:56.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-23T08:13:56.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:13:56.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-23T08:13:56.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-23T08:13:56.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-23T08:13:56.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:13:56.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:13:56.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:13:56.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-23T08:13:56.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-23T08:13:56.042 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:13:56.042 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:56.042 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:56.042 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:56.042 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.042 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.042 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:13:56.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:13:56.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:13:56.072 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:13:56.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:56.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:56.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:13:56.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:13:56.074 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:56.074 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:56.074 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:56.074 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:56.074 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.074 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.075 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46012/ceph-mon.a.asok 2026-02-23T08:13:56.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:56.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46012/ceph-mon.a.asok config get fsid 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:56.132 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:56.133 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:56.133 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.134 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.134 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46012/ceph-mon.a.asok 2026-02-23T08:13:56.134 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:13:56.134 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.46012/ceph-mon.a.asok config get mon_host 2026-02-23T08:13:56.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-23T08:13:56.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-23T08:13:56.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:13:56.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:13:56.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:13:56.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-23T08:13:56.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:13:56.304 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:13:56.304 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:56.304 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:56.304 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:56.304 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.305 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.305 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:13:56.305 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:56.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:13:56.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-23T08:13:56.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:13:56.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:13:56.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:13:56.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46012/ceph-mon.a.asok 2026-02-23T08:13:56.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-23T08:13:56.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.46012/ceph-mon.a.asok log flush 2026-02-23T08:13:56.380 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:13:56.387 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-23T08:13:56.389 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:56.056+0000 7fe92e60dec0 0 load: jerasure load: lrc 2026-02-23T08:13:56.389 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-23T08:13:56.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:13:56.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-23T08:13:56.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:13:56.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-23T08:13:56.396 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:13:56.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=50b61f6a-74b7-45cf-a336-f94d07bb67f4 2026-02-23T08:13:56.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 50b61f6a-74b7-45cf-a336-f94d07bb67f4' 2026-02-23T08:13:56.398 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 50b61f6a-74b7-45cf-a336-f94d07bb67f4 2026-02-23T08:13:56.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:13:56.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEDJxpgvorGBAANEMwiXk1QPAixyLbqVMg+Q== 2026-02-23T08:13:56.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEDJxpgvorGBAANEMwiXk1QPAixyLbqVMg+Q=="}' 2026-02-23T08:13:56.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 50b61f6a-74b7-45cf-a336-f94d07bb67f4 -i td/test-erasure-code/0/new.json 2026-02-23T08:13:56.521 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:56.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-23T08:13:56.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEDJxpgvorGBAANEMwiXk1QPAixyLbqVMg+Q== --osd-uuid 50b61f6a-74b7-45cf-a336-f94d07bb67f4 2026-02-23T08:13:56.547 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:56.540+0000 7f5bd7f1da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:56.548 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:56.544+0000 7f5bd7f1da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:56.550 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:56.544+0000 7f5bd7f1da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:56.551 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:56.544+0000 7f5bd7f1da40 -1 bdev(0x55a194d08400 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:13:56.551 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:56.544+0000 7f5bd7f1da40 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-23T08:13:57.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-23T08:13:57.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:13:57.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:13:57.396 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:13:57.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:13:57.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:13:57.517 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:13:57.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:13:57.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:13:57.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:13:57.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:13:57.539 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:57.536+0000 7f4fc4e5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:57.541 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:57.536+0000 7f4fc4e5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:57.542 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:57.536+0000 7f4fc4e5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:13:57.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:13:57.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:13:57.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:13:57.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:13:57.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:13:57.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:57.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:13:57.634 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:13:57.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:57.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:57.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:57.976 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:57.972+0000 7f4fc4e5aa40 -1 Falling back to public interface 2026-02-23T08:13:58.671 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:13:58.668+0000 7f4fc4e5aa40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:13:58.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:58.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:58.745 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:13:58.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:13:58.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:58.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:13:58.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:13:59.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:13:59.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:13:59.965 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:13:59.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:13:59.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:13:59.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:14:00.187 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3976848144,v1:127.0.0.1:6801/3976848144] [v2:127.0.0.1:6802/3976848144,v1:127.0.0.1:6803/3976848144] exists,up 50b61f6a-74b7-45cf-a336-f94d07bb67f4 2026-02-23T08:14:00.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-23T08:14:00.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:00.192 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:00.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:00.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:00.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:00.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:00.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-23T08:14:00.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:00.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=75a2974d-a9ec-4f1a-809c-61d36c6dab35 2026-02-23T08:14:00.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 75a2974d-a9ec-4f1a-809c-61d36c6dab35' 2026-02-23T08:14:00.195 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 75a2974d-a9ec-4f1a-809c-61d36c6dab35 2026-02-23T08:14:00.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:00.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIDJxpbo8wDBAAB7LviVowi8yulXnCfyA8Aw== 2026-02-23T08:14:00.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIDJxpbo8wDBAAB7LviVowi8yulXnCfyA8Aw=="}' 2026-02-23T08:14:00.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 75a2974d-a9ec-4f1a-809c-61d36c6dab35 -i td/test-erasure-code/1/new.json 2026-02-23T08:14:00.426 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:00.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-23T08:14:00.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIDJxpbo8wDBAAB7LviVowi8yulXnCfyA8Aw== --osd-uuid 75a2974d-a9ec-4f1a-809c-61d36c6dab35 2026-02-23T08:14:00.460 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:00.456+0000 7f22a9dd8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:00.461 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:00.456+0000 7f22a9dd8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:00.463 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:00.460+0000 7f22a9dd8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:00.463 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:00.460+0000 7f22a9dd8a40 -1 bdev(0x55bc7bdca400 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:00.463 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:00.460+0000 7f22a9dd8a40 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-23T08:14:01.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-23T08:14:01.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:01.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:14:01.132 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:14:01.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:01.432 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:14:01.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:14:01.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:01.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:01.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:01.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:01.454 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:01.448+0000 7f9bd1718a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:01.456 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:01.452+0000 7f9bd1718a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:01.457 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:01.452+0000 7f9bd1718a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:14:01.856 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:01.852+0000 7f9bd1718a40 -1 Falling back to public interface 2026-02-23T08:14:01.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:02.541 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:02.536+0000 7f9bd1718a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:14:02.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:02.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:02.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:02.873 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:02.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:02.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:14:03.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:04.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:04.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:04.090 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:14:04.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:14:04.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:04.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:14:04.303 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/591971123,v1:127.0.0.1:6811/591971123] [v2:127.0.0.1:6812/591971123,v1:127.0.0.1:6813/591971123] exists,up 75a2974d-a9ec-4f1a-809c-61d36c6dab35 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-23T08:14:04.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:04.306 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:04.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-23T08:14:04.308 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:04.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dd90898e-2a8b-4315-96b0-1bec59337607 2026-02-23T08:14:04.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 dd90898e-2a8b-4315-96b0-1bec59337607' 2026-02-23T08:14:04.308 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 dd90898e-2a8b-4315-96b0-1bec59337607 2026-02-23T08:14:04.308 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:04.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBMDJxpDCMUExAAQqtWbMKhPMoV+Ucb4H30MQ== 2026-02-23T08:14:04.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBMDJxpDCMUExAAQqtWbMKhPMoV+Ucb4H30MQ=="}' 2026-02-23T08:14:04.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dd90898e-2a8b-4315-96b0-1bec59337607 -i td/test-erasure-code/2/new.json 2026-02-23T08:14:04.547 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:14:04.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-23T08:14:04.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBMDJxpDCMUExAAQqtWbMKhPMoV+Ucb4H30MQ== --osd-uuid dd90898e-2a8b-4315-96b0-1bec59337607 2026-02-23T08:14:04.573 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:04.568+0000 7f7977153a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:04.574 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:04.568+0000 7f7977153a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:04.575 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:04.572+0000 7f7977153a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:04.575 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:04.572+0000 7f7977153a40 -1 bdev(0x55cab0a52400 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:04.575 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:04.572+0000 7f7977153a40 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-23T08:14:05.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-23T08:14:05.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:05.202 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:14:05.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:14:05.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:05.498 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:14:05.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:14:05.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:05.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:05.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:05.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:05.513 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:05.508+0000 7f0adc95fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:05.515 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:05.512+0000 7f0adc95fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:05.516 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:05.512+0000 7f0adc95fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:05.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:14:05.864 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:05.860+0000 7f0adc95fa40 -1 Falling back to public interface 2026-02-23T08:14:05.932 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:06.537 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:06.532+0000 7f0adc95fa40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:14:06.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:06.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:06.934 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:06.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:06.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:06.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:14:07.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:08.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:08.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:08.145 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:14:08.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:14:08.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:08.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:14:08.364 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1238216776,v1:127.0.0.1:6819/1238216776] [v2:127.0.0.1:6820/1238216776,v1:127.0.0.1:6821/1238216776] exists,up dd90898e-2a8b-4315-96b0-1bec59337607 2026-02-23T08:14:08.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-23T08:14:08.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:08.369 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:08.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:08.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:08.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:08.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-23T08:14:08.371 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:08.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f8017f9d-8067-41af-8023-ca42895cb644 2026-02-23T08:14:08.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 f8017f9d-8067-41af-8023-ca42895cb644' 2026-02-23T08:14:08.372 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 f8017f9d-8067-41af-8023-ca42895cb644 2026-02-23T08:14:08.372 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:08.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBQDJxpgeyfFhAA838YSgNVLtwKtLcQ/TYrbw== 2026-02-23T08:14:08.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBQDJxpgeyfFhAA838YSgNVLtwKtLcQ/TYrbw=="}' 2026-02-23T08:14:08.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f8017f9d-8067-41af-8023-ca42895cb644 -i td/test-erasure-code/3/new.json 2026-02-23T08:14:08.599 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:14:08.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-23T08:14:08.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBQDJxpgeyfFhAA838YSgNVLtwKtLcQ/TYrbw== --osd-uuid f8017f9d-8067-41af-8023-ca42895cb644 2026-02-23T08:14:08.635 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:08.632+0000 7f67ed5c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:08.637 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:08.632+0000 7f67ed5c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:08.638 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:08.632+0000 7f67ed5c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:08.638 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:08.632+0000 7f67ed5c9a40 -1 bdev(0x558163a38400 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:08.638 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:08.632+0000 7f67ed5c9a40 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-23T08:14:09.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-23T08:14:09.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:09.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:14:09.251 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:14:09.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:09.553 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:14:09.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:14:09.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:09.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:09.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:09.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:09.568 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:09.564+0000 7fd4468efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:09.570 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:09.564+0000 7fd4468efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:09.571 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:09.568+0000 7fd4468efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:09.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:14:09.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:09.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:14:09.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:09.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:09.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:09.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:09.766 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:09.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:09.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:14:09.952 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:09.948+0000 7fd4468efa40 -1 Falling back to public interface 2026-02-23T08:14:09.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:10.505 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:10.500+0000 7fd4468efa40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:14:10.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:10.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:10.983 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:10.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:10.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:10.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:14:11.209 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3858876587,v1:127.0.0.1:6827/3858876587] [v2:127.0.0.1:6828/3858876587,v1:127.0.0.1:6829/3858876587] exists,up f8017f9d-8067-41af-8023-ca42895cb644 2026-02-23T08:14:11.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-23T08:14:11.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:11.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:11.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:11.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:11.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:11.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-23T08:14:11.216 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:11.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2da9d975-8bc9-41b1-9e17-4d0f3903f053 2026-02-23T08:14:11.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 2da9d975-8bc9-41b1-9e17-4d0f3903f053' 2026-02-23T08:14:11.217 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 2da9d975-8bc9-41b1-9e17-4d0f3903f053 2026-02-23T08:14:11.217 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:11.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBTDJxpHjR8DRAAcrFbZqBh+LtnrnnLBL2c/A== 2026-02-23T08:14:11.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBTDJxpHjR8DRAAcrFbZqBh+LtnrnnLBL2c/A=="}' 2026-02-23T08:14:11.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2da9d975-8bc9-41b1-9e17-4d0f3903f053 -i td/test-erasure-code/4/new.json 2026-02-23T08:14:11.456 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:14:11.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-23T08:14:11.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTDJxpHjR8DRAAcrFbZqBh+LtnrnnLBL2c/A== --osd-uuid 2da9d975-8bc9-41b1-9e17-4d0f3903f053 2026-02-23T08:14:11.483 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:11.480+0000 7f88f7770a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:11.485 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:11.480+0000 7f88f7770a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:11.486 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:11.480+0000 7f88f7770a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:11.486 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:11.480+0000 7f88f7770a40 -1 bdev(0x55e4bba98400 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:11.486 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:11.480+0000 7f88f7770a40 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-23T08:14:12.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-23T08:14:12.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:12.162 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:14:12.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:14:12.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:12.465 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:14:12.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:14:12.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:12.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:12.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:12.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:12.481 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:12.476+0000 7facc08cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:12.483 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:12.480+0000 7facc08cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:12.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:12.480+0000 7facc08cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:12.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:14:12.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:12.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:14:12.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:12.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:12.682 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:12.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:12.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:12.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:12.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:14:12.840 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:12.836+0000 7facc08cca40 -1 Falling back to public interface 2026-02-23T08:14:12.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:13.433 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:13.428+0000 7facc08cca40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:14:13.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:13.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:13.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:13.900 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:13.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:13.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:14:14.104 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1991490873,v1:127.0.0.1:6835/1991490873] [v2:127.0.0.1:6836/1991490873,v1:127.0.0.1:6837/1991490873] exists,up 2da9d975-8bc9-41b1-9e17-4d0f3903f053 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:14.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:14.109 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:14.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:14.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:14.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:14.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:14.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-23T08:14:14.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:14.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=060155e1-01a7-4a6c-87d7-b2b4f55eaf2f 2026-02-23T08:14:14.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 060155e1-01a7-4a6c-87d7-b2b4f55eaf2f' 2026-02-23T08:14:14.112 INFO:tasks.workunit.client.0.trial186.stdout:add osd5 060155e1-01a7-4a6c-87d7-b2b4f55eaf2f 2026-02-23T08:14:14.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:14.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBWDJxpEJFDBxAAc/1na8IBF5NoYwChXWvTQQ== 2026-02-23T08:14:14.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBWDJxpEJFDBxAAc/1na8IBF5NoYwChXWvTQQ=="}' 2026-02-23T08:14:14.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 060155e1-01a7-4a6c-87d7-b2b4f55eaf2f -i td/test-erasure-code/5/new.json 2026-02-23T08:14:14.343 INFO:tasks.workunit.client.0.trial186.stdout:5 2026-02-23T08:14:14.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-23T08:14:14.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBWDJxpEJFDBxAAc/1na8IBF5NoYwChXWvTQQ== --osd-uuid 060155e1-01a7-4a6c-87d7-b2b4f55eaf2f 2026-02-23T08:14:14.379 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:14.376+0000 7f0d3cbd6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:14.380 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:14.376+0000 7f0d3cbd6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:14.381 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:14.376+0000 7f0d3cbd6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:14.382 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:14.376+0000 7f0d3cbd6a40 -1 bdev(0x557fefc8a400 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:14.382 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:14.376+0000 7f0d3cbd6a40 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-23T08:14:15.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-23T08:14:15.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:15.075 INFO:tasks.workunit.client.0.trial186.stdout:adding osd5 key to auth repository 2026-02-23T08:14:15.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-23T08:14:15.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:15.377 INFO:tasks.workunit.client.0.trial186.stdout:start osd.5 2026-02-23T08:14:15.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-23T08:14:15.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:15.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:15.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:15.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:15.400 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:15.396+0000 7f5fbed51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:15.402 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:15.396+0000 7f5fbed51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:15.403 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:15.400+0000 7f5fbed51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:15.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-23T08:14:15.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:15.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-23T08:14:15.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:15.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:15.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:15.598 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:15.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:15.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:15.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:14:15.804 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:15.800+0000 7f5fbed51a40 -1 Falling back to public interface 2026-02-23T08:14:15.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:16.360 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:16.356+0000 7f5fbed51a40 -1 osd.5 0 log_to_monitors true 2026-02-23T08:14:16.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:16.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:16.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:16.817 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:16.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:16.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:14:17.035 INFO:tasks.workunit.client.0.trial186.stdout:osd.5 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1582752130,v1:127.0.0.1:6843/1582752130] [v2:127.0.0.1:6844/1582752130,v1:127.0.0.1:6845/1582752130] exists,up 060155e1-01a7-4a6c-87d7-b2b4f55eaf2f 2026-02-23T08:14:17.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:17.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:17.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:17.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=36ce2271-f4a8-41ef-8ec1-17c1156151c9 2026-02-23T08:14:17.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 36ce2271-f4a8-41ef-8ec1-17c1156151c9' 2026-02-23T08:14:17.044 INFO:tasks.workunit.client.0.trial186.stdout:add osd6 36ce2271-f4a8-41ef-8ec1-17c1156151c9 2026-02-23T08:14:17.044 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:17.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZDJxpcUsmAxAA1U7jOb+jPHrjFJ2WYOpJ6A== 2026-02-23T08:14:17.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZDJxpcUsmAxAA1U7jOb+jPHrjFJ2WYOpJ6A=="}' 2026-02-23T08:14:17.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 36ce2271-f4a8-41ef-8ec1-17c1156151c9 -i td/test-erasure-code/6/new.json 2026-02-23T08:14:17.284 INFO:tasks.workunit.client.0.trial186.stdout:6 2026-02-23T08:14:17.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-23T08:14:17.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZDJxpcUsmAxAA1U7jOb+jPHrjFJ2WYOpJ6A== --osd-uuid 36ce2271-f4a8-41ef-8ec1-17c1156151c9 2026-02-23T08:14:17.311 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:17.308+0000 7f5596827a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:17.313 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:17.308+0000 7f5596827a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:17.314 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:17.308+0000 7f5596827a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:17.314 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:17.308+0000 7f5596827a40 -1 bdev(0x556bca5c2400 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:17.314 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:17.308+0000 7f5596827a40 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-23T08:14:17.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-23T08:14:17.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:17.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-23T08:14:17.948 INFO:tasks.workunit.client.0.trial186.stdout:adding osd6 key to auth repository 2026-02-23T08:14:17.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:18.250 INFO:tasks.workunit.client.0.trial186.stdout:start osd.6 2026-02-23T08:14:18.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-23T08:14:18.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:18.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:18.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:18.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:18.274 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:18.268+0000 7fe7c39fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:18.275 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:18.272+0000 7fe7c39fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:18.276 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:18.272+0000 7fe7c39fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:18.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-23T08:14:18.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:18.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-23T08:14:18.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:18.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:18.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:18.468 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:18.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:18.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:18.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:14:18.644 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:18.640+0000 7fe7c39fca40 -1 Falling back to public interface 2026-02-23T08:14:18.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:19.220 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:19.216+0000 7fe7c39fca40 -1 osd.6 0 log_to_monitors true 2026-02-23T08:14:19.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:19.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:19.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:19.687 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:19.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:19.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:14:19.896 INFO:tasks.workunit.client.0.trial186.stdout:osd.6 up in weight 1 up_from 31 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/928426714,v1:127.0.0.1:6851/928426714] [v2:127.0.0.1:6852/928426714,v1:127.0.0.1:6853/928426714] exists,up 36ce2271-f4a8-41ef-8ec1-17c1156151c9 2026-02-23T08:14:19.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:19.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:19.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:19.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:19.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-23T08:14:19.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:19.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-23T08:14:19.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:19.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:19.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-23T08:14:19.904 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:19.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=00bf780a-9a6a-485c-a9ad-8d702c1fda29 2026-02-23T08:14:19.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 00bf780a-9a6a-485c-a9ad-8d702c1fda29' 2026-02-23T08:14:19.905 INFO:tasks.workunit.client.0.trial186.stdout:add osd7 00bf780a-9a6a-485c-a9ad-8d702c1fda29 2026-02-23T08:14:19.905 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:19.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBbDJxpFv1vNhAAjwZxgvzAae4ERuO50zApQg== 2026-02-23T08:14:19.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBbDJxpFv1vNhAAjwZxgvzAae4ERuO50zApQg=="}' 2026-02-23T08:14:19.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 00bf780a-9a6a-485c-a9ad-8d702c1fda29 -i td/test-erasure-code/7/new.json 2026-02-23T08:14:20.142 INFO:tasks.workunit.client.0.trial186.stdout:7 2026-02-23T08:14:20.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-23T08:14:20.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBbDJxpFv1vNhAAjwZxgvzAae4ERuO50zApQg== --osd-uuid 00bf780a-9a6a-485c-a9ad-8d702c1fda29 2026-02-23T08:14:20.169 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:20.164+0000 7f2567740a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:20.171 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:20.168+0000 7f2567740a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:20.172 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:20.168+0000 7f2567740a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:20.172 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:20.168+0000 7f2567740a40 -1 bdev(0x56182069e400 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:20.172 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:20.168+0000 7f2567740a40 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-23T08:14:20.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-23T08:14:20.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:20.850 INFO:tasks.workunit.client.0.trial186.stdout:adding osd7 key to auth repository 2026-02-23T08:14:20.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-23T08:14:20.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:21.150 INFO:tasks.workunit.client.0.trial186.stdout:start osd.7 2026-02-23T08:14:21.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-23T08:14:21.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:21.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:21.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:21.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:21.167 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:21.164+0000 7fd958880a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:21.168 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:21.164+0000 7fd958880a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:21.169 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:21.164+0000 7fd958880a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:21.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-23T08:14:21.528 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:21.524+0000 7fd958880a40 -1 Falling back to public interface 2026-02-23T08:14:21.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:22.137 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:22.132+0000 7fd958880a40 -1 osd.7 0 log_to_monitors true 2026-02-23T08:14:22.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:22.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:22.590 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:22.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:22.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:22.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-23T08:14:22.802 INFO:tasks.workunit.client.0.trial186.stdout:osd.7 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/3053704885,v1:127.0.0.1:6859/3053704885] [v2:127.0.0.1:6860/3053704885,v1:127.0.0.1:6861/3053704885] exists,up 00bf780a-9a6a-485c-a9ad-8d702c1fda29 2026-02-23T08:14:22.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:22.804 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:22.805 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:22.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-23T08:14:22.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:22.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f4416dc8-676a-4f5c-b009-6a67cfc4072e 2026-02-23T08:14:22.807 INFO:tasks.workunit.client.0.trial186.stdout:add osd8 f4416dc8-676a-4f5c-b009-6a67cfc4072e 2026-02-23T08:14:22.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 f4416dc8-676a-4f5c-b009-6a67cfc4072e' 2026-02-23T08:14:22.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:22.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBeDJxplrftMBAAdXfeCDzRTMkbW/9WcZwB2A== 2026-02-23T08:14:22.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBeDJxplrftMBAAdXfeCDzRTMkbW/9WcZwB2A=="}' 2026-02-23T08:14:22.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f4416dc8-676a-4f5c-b009-6a67cfc4072e -i td/test-erasure-code/8/new.json 2026-02-23T08:14:23.040 INFO:tasks.workunit.client.0.trial186.stdout:8 2026-02-23T08:14:23.049 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-23T08:14:23.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBeDJxplrftMBAAdXfeCDzRTMkbW/9WcZwB2A== --osd-uuid f4416dc8-676a-4f5c-b009-6a67cfc4072e 2026-02-23T08:14:23.066 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:23.060+0000 7f2eb404fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:23.067 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:23.064+0000 7f2eb404fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:23.068 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:23.064+0000 7f2eb404fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:23.069 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:23.064+0000 7f2eb404fa40 -1 bdev(0x556d1afe6400 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:23.069 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:23.064+0000 7f2eb404fa40 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-23T08:14:23.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-23T08:14:23.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:23.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-23T08:14:23.746 INFO:tasks.workunit.client.0.trial186.stdout:adding osd8 key to auth repository 2026-02-23T08:14:23.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:24.045 INFO:tasks.workunit.client.0.trial186.stdout:start osd.8 2026-02-23T08:14:24.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-23T08:14:24.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:24.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:24.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:24.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:24.069 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:24.064+0000 7f8f2dea0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:24.071 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:24.068+0000 7f8f2dea0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:24.072 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:24.068+0000 7f8f2dea0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:24.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-23T08:14:24.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:24.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-23T08:14:24.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:24.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:24.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:24.267 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:24.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:24.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:24.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-23T08:14:24.432 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:24.428+0000 7f8f2dea0a40 -1 Falling back to public interface 2026-02-23T08:14:24.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:25.083 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:25.080+0000 7f8f2dea0a40 -1 osd.8 0 log_to_monitors true 2026-02-23T08:14:25.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:25.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:25.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:25.478 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:25.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:25.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-23T08:14:25.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:26.690 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:26.690 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:26.690 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:14:26.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:14:26.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:26.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-23T08:14:26.903 INFO:tasks.workunit.client.0.trial186.stdout:osd.8 up in weight 1 up_from 41 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/3010093663,v1:127.0.0.1:6867/3010093663] [v2:127.0.0.1:6868/3010093663,v1:127.0.0.1:6869/3010093663] exists,up f4416dc8-676a-4f5c-b009-6a67cfc4072e 2026-02-23T08:14:26.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:26.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:26.908 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:26.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:26.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:26.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-23T08:14:26.910 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:26.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4472fe7f-4301-4f30-887f-b0cbf5ab6b82 2026-02-23T08:14:26.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 4472fe7f-4301-4f30-887f-b0cbf5ab6b82' 2026-02-23T08:14:26.911 INFO:tasks.workunit.client.0.trial186.stdout:add osd9 4472fe7f-4301-4f30-887f-b0cbf5ab6b82 2026-02-23T08:14:26.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:26.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBiDJxp6r/rNhAAanRpl7Jcn6vMpqzKD9desQ== 2026-02-23T08:14:26.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBiDJxp6r/rNhAAanRpl7Jcn6vMpqzKD9desQ=="}' 2026-02-23T08:14:26.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4472fe7f-4301-4f30-887f-b0cbf5ab6b82 -i td/test-erasure-code/9/new.json 2026-02-23T08:14:27.138 INFO:tasks.workunit.client.0.trial186.stdout:9 2026-02-23T08:14:27.147 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-23T08:14:27.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBiDJxp6r/rNhAAanRpl7Jcn6vMpqzKD9desQ== --osd-uuid 4472fe7f-4301-4f30-887f-b0cbf5ab6b82 2026-02-23T08:14:27.172 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:27.168+0000 7f60c76f3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:27.173 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:27.168+0000 7f60c76f3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:27.174 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:27.172+0000 7f60c76f3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:27.175 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:27.172+0000 7f60c76f3a40 -1 bdev(0x55dbc6b58400 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:27.175 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:27.172+0000 7f60c76f3a40 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-23T08:14:27.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-23T08:14:27.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:27.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-23T08:14:27.807 INFO:tasks.workunit.client.0.trial186.stdout:adding osd9 key to auth repository 2026-02-23T08:14:27.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:28.103 INFO:tasks.workunit.client.0.trial186.stdout:start osd.9 2026-02-23T08:14:28.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-23T08:14:28.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:28.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:28.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:28.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:28.119 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:28.116+0000 7f5d61e33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:28.120 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:28.116+0000 7f5d61e33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:28.121 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:28.116+0000 7f5d61e33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:28.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-23T08:14:28.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:28.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-23T08:14:28.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:28.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:28.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:28.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:28.324 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:28.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:28.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-23T08:14:28.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:28.480+0000 7f5d61e33a40 -1 Falling back to public interface 2026-02-23T08:14:28.534 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:29.128 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:29.124+0000 7f5d61e33a40 -1 osd.9 0 log_to_monitors true 2026-02-23T08:14:29.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:29.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:29.536 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:29.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:29.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:29.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-23T08:14:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:30.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:30.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:30.745 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:14:30.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:14:30.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:30.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-23T08:14:30.958 INFO:tasks.workunit.client.0.trial186.stdout:osd.9 up in weight 1 up_from 45 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/596642028,v1:127.0.0.1:6875/596642028] [v2:127.0.0.1:6876/596642028,v1:127.0.0.1:6877/596642028] exists,up 4472fe7f-4301-4f30-887f-b0cbf5ab6b82 2026-02-23T08:14:30.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:30.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:30.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:30.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-23T08:14:30.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-23T08:14:30.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:30.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-23T08:14:30.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:14:30.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8735f9b7-ab9e-459a-bda1-085299e3012c 2026-02-23T08:14:30.966 INFO:tasks.workunit.client.0.trial186.stdout:add osd10 8735f9b7-ab9e-459a-bda1-085299e3012c 2026-02-23T08:14:30.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 8735f9b7-ab9e-459a-bda1-085299e3012c' 2026-02-23T08:14:30.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:14:30.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBmDJxpImsyOhAAyj8KzK1WRJOBS/Z5YXRfKQ== 2026-02-23T08:14:30.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBmDJxpImsyOhAAyj8KzK1WRJOBS/Z5YXRfKQ=="}' 2026-02-23T08:14:30.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8735f9b7-ab9e-459a-bda1-085299e3012c -i td/test-erasure-code/10/new.json 2026-02-23T08:14:31.200 INFO:tasks.workunit.client.0.trial186.stdout:10 2026-02-23T08:14:31.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-23T08:14:31.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBmDJxpImsyOhAAyj8KzK1WRJOBS/Z5YXRfKQ== --osd-uuid 8735f9b7-ab9e-459a-bda1-085299e3012c 2026-02-23T08:14:31.227 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:31.224+0000 7f97fe4dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:31.228 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:31.224+0000 7f97fe4dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:31.229 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:31.224+0000 7f97fe4dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:31.230 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:31.224+0000 7f97fe4dda40 -1 bdev(0x5577d94cc400 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-23T08:14:31.230 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:31.224+0000 7f97fe4dda40 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-23T08:14:31.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-23T08:14:31.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:14:31.906 INFO:tasks.workunit.client.0.trial186.stdout:adding osd10 key to auth repository 2026-02-23T08:14:31.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-23T08:14:31.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:14:32.197 INFO:tasks.workunit.client.0.trial186.stdout:start osd.10 2026-02-23T08:14:32.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-23T08:14:32.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:32.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:14:32.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:14:32.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:14:32.212 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:32.208+0000 7f7e21b2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:32.214 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:32.208+0000 7f7e21b2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:32.215 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:32.212+0000 7f7e21b2ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:32.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-23T08:14:32.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:32.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-23T08:14:32.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:32.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:32.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:32.403 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:14:32.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:32.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:32.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-23T08:14:32.568 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:32.564+0000 7f7e21b2ea40 -1 Falling back to public interface 2026-02-23T08:14:32.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:33.208 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:33.204+0000 7f7e21b2ea40 -1 osd.10 0 log_to_monitors true 2026-02-23T08:14:33.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:33.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:33.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:33.628 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:14:33.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:33.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-23T08:14:33.845 INFO:tasks.workunit.client.0.trial186.stdout:osd.10 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/1301713610,v1:127.0.0.1:6883/1301713610] [v2:127.0.0.1:6884/1301713610,v1:127.0.0.1:6885/1301713610] exists,up 8735f9b7-ab9e-459a-bda1-085299e3012c 2026-02-23T08:14:33.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:33.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:33.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:33.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-23T08:14:33.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-23T08:14:34.054 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' does not exist 2026-02-23T08:14:34.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-23T08:14:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:14:34.346 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:14:34.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:14:35.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-23T08:14:35.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-23T08:14:35.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:14:35.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:14:35.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:14:35.606 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:14:35.607 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:14:35.607 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:14:35.607 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:14:35.607 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:14:35.607 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:14:35.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:14:35.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:14:35.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:14:35.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:14:35.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:14:35.709 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:14:35.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:14:35.925 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:14:35.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:14:35.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:35.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:14:36.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-23T08:14:36.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-23T08:14:36.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-23T08:14:36.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.007 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:14:36.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705673 2026-02-23T08:14:36.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705673 2026-02-23T08:14:36.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673' 2026-02-23T08:14:36.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.086 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:14:36.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-23T08:14:36.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-23T08:14:36.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856' 2026-02-23T08:14:36.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.168 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:14:36.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444039 2026-02-23T08:14:36.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444039 2026-02-23T08:14:36.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039' 2026-02-23T08:14:36.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.247 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:14:36.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280519 2026-02-23T08:14:36.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280519 2026-02-23T08:14:36.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039 4-94489280519' 2026-02-23T08:14:36.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.324 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:14:36.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116998 2026-02-23T08:14:36.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116998 2026-02-23T08:14:36.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039 4-94489280519 5-115964116998' 2026-02-23T08:14:36.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:14:36.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986182 2026-02-23T08:14:36.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986182 2026-02-23T08:14:36.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039 4-94489280519 5-115964116998 6-133143986182' 2026-02-23T08:14:36.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:14:36.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822661 2026-02-23T08:14:36.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822661 2026-02-23T08:14:36.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039 4-94489280519 5-115964116998 6-133143986182 7-154618822661' 2026-02-23T08:14:36.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.565 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:14:36.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659141 2026-02-23T08:14:36.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659141 2026-02-23T08:14:36.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039 4-94489280519 5-115964116998 6-133143986182 7-154618822661 8-176093659141' 2026-02-23T08:14:36.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:14:36.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528324 2026-02-23T08:14:36.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528324 2026-02-23T08:14:36.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039 4-94489280519 5-115964116998 6-133143986182 7-154618822661 8-176093659141 9-193273528324' 2026-02-23T08:14:36.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:36.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:14:36.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-23T08:14:36.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-23T08:14:36.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856 3-73014444039 4-94489280519 5-115964116998 6-133143986182 7-154618822661 8-176093659141 9-193273528324 10-214748364803' 2026-02-23T08:14:36.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:36.809 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-23T08:14:36.809 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:36.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:14:36.810 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-23T08:14:36.810 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:36.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-23T08:14:36.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-23T08:14:36.812 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 21474836490 2026-02-23T08:14:36.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:14:37.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-23T08:14:37.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:37.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705673 2026-02-23T08:14:37.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:37.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:14:37.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705673 2026-02-23T08:14:37.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:37.033 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705673 2026-02-23T08:14:37.033 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-23T08:14:37.033 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 38654705673 2026-02-23T08:14:37.033 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:14:37.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705673 -lt 38654705673 2026-02-23T08:14:37.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:37.247 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574856 2026-02-23T08:14:37.247 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:37.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:14:37.248 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574856 2026-02-23T08:14:37.248 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:37.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-23T08:14:37.250 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 55834574856 2026-02-23T08:14:37.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574856' 2026-02-23T08:14:37.251 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:14:37.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-23T08:14:37.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:37.478 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-73014444039 2026-02-23T08:14:37.478 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:37.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:14:37.480 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-73014444039 2026-02-23T08:14:37.480 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:37.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444039 2026-02-23T08:14:37.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 73014444039' 2026-02-23T08:14:37.482 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 73014444039 2026-02-23T08:14:37.482 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:14:37.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-23T08:14:37.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:37.712 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280519 2026-02-23T08:14:37.712 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:37.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:14:37.714 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280519 2026-02-23T08:14:37.714 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:37.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280519 2026-02-23T08:14:37.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280519' 2026-02-23T08:14:37.716 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 94489280519 2026-02-23T08:14:37.716 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:14:37.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280519 -lt 94489280519 2026-02-23T08:14:37.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:37.931 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-115964116998 2026-02-23T08:14:37.931 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:37.932 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:14:37.933 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-115964116998 2026-02-23T08:14:37.933 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:37.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116998 2026-02-23T08:14:37.934 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 115964116998 2026-02-23T08:14:37.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 115964116998' 2026-02-23T08:14:37.935 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:14:38.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116998 -lt 115964116998 2026-02-23T08:14:38.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:38.158 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-133143986182 2026-02-23T08:14:38.158 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:38.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:14:38.160 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-133143986182 2026-02-23T08:14:38.160 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:38.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=133143986182 2026-02-23T08:14:38.161 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 133143986182 2026-02-23T08:14:38.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 133143986182' 2026-02-23T08:14:38.162 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:14:38.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 133143986182 -lt 133143986182 2026-02-23T08:14:38.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:38.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822661 2026-02-23T08:14:38.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:38.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:14:38.383 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822661 2026-02-23T08:14:38.383 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:38.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822661 2026-02-23T08:14:38.385 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.7 seq 154618822661 2026-02-23T08:14:38.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822661' 2026-02-23T08:14:38.385 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:14:38.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822661 -lt 154618822661 2026-02-23T08:14:38.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:38.613 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659141 2026-02-23T08:14:38.613 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:38.614 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:14:38.615 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659141 2026-02-23T08:14:38.615 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:38.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659141 2026-02-23T08:14:38.617 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.8 seq 176093659141 2026-02-23T08:14:38.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659141' 2026-02-23T08:14:38.617 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:14:38.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659141 -lt 176093659141 2026-02-23T08:14:38.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:38.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-193273528324 2026-02-23T08:14:38.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:38.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:14:38.845 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-193273528324 2026-02-23T08:14:38.845 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:38.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=193273528324 2026-02-23T08:14:38.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 193273528324' 2026-02-23T08:14:38.847 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.9 seq 193273528324 2026-02-23T08:14:38.847 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:14:39.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 193273528324 -lt 193273528324 2026-02-23T08:14:39.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:39.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364803 2026-02-23T08:14:39.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:39.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:14:39.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364803 2026-02-23T08:14:39.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:39.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364803 2026-02-23T08:14:39.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364803' 2026-02-23T08:14:39.077 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.10 seq 214748364803 2026-02-23T08:14:39.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:14:39.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-23T08:14:39.304 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:14:39.305 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:14:39.305 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:14:39.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-23T08:14:39.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:14:39.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:14:39.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:14:39.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:14:39.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:14:39.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:14:39.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:14:39.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-23T08:14:39.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:14:39.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:14:39.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:14:40.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-23T08:14:40.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:14:40.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.46012/ceph-osd.0.asok 2026-02-23T08:14:40.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-23T08:14:40.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.46012/ceph-osd.0.asok log flush 2026-02-23T08:14:40.167 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:14:40.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-23T08:14:40.174 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:13:57.976+0000 7f4fc4e5aa40 0 load: jerasure load: lrc 2026-02-23T08:14:40.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-23T08:14:40.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-23T08:14:40.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-23T08:14:40.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-23T08:14:40.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-23T08:14:40.809 INFO:tasks.workunit.client.0.trial186.stderr:pool 'ecpool' already exists 2026-02-23T08:14:40.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:14:41.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:14:41.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:14:41.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:14:41.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:14:41.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:14:41.823 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:14:41.823 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:14:41.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:14:41.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:14:41.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:14:41.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:14:41.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:14:41.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:14:41.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:14:41.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:14:41.924 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:14:42.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:14:42.133 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:14:42.134 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:14:42.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:14:42.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.134 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:14:42.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-23T08:14:42.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-23T08:14:42.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493' 2026-02-23T08:14:42.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:14:42.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705676 2026-02-23T08:14:42.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705676 2026-02-23T08:14:42.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676' 2026-02-23T08:14:42.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.291 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:14:42.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574859 2026-02-23T08:14:42.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574859 2026-02-23T08:14:42.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859' 2026-02-23T08:14:42.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.372 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:14:42.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444043 2026-02-23T08:14:42.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444043 2026-02-23T08:14:42.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043' 2026-02-23T08:14:42.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.449 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:14:42.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280522 2026-02-23T08:14:42.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280522 2026-02-23T08:14:42.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043 4-94489280522' 2026-02-23T08:14:42.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:14:42.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117002 2026-02-23T08:14:42.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117002 2026-02-23T08:14:42.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043 4-94489280522 5-115964117002' 2026-02-23T08:14:42.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.605 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:14:42.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986185 2026-02-23T08:14:42.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986185 2026-02-23T08:14:42.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043 4-94489280522 5-115964117002 6-133143986185' 2026-02-23T08:14:42.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.685 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:14:42.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822664 2026-02-23T08:14:42.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822664 2026-02-23T08:14:42.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043 4-94489280522 5-115964117002 6-133143986185 7-154618822664' 2026-02-23T08:14:42.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:14:42.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659144 2026-02-23T08:14:42.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659144 2026-02-23T08:14:42.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043 4-94489280522 5-115964117002 6-133143986185 7-154618822664 8-176093659144' 2026-02-23T08:14:42.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.845 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:14:42.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528327 2026-02-23T08:14:42.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528327 2026-02-23T08:14:42.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043 4-94489280522 5-115964117002 6-133143986185 7-154618822664 8-176093659144 9-193273528327' 2026-02-23T08:14:42.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:42.926 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:14:43.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364806 2026-02-23T08:14:43.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364806 2026-02-23T08:14:43.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574859 3-73014444043 4-94489280522 5-115964117002 6-133143986185 7-154618822664 8-176093659144 9-193273528327 10-214748364806' 2026-02-23T08:14:43.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:43.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836493 2026-02-23T08:14:43.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:43.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:14:43.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-23T08:14:43.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:43.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836493 2026-02-23T08:14:43.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-23T08:14:43.006 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 21474836493 2026-02-23T08:14:43.006 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:14:43.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-23T08:14:43.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:43.234 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705676 2026-02-23T08:14:43.234 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:43.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:14:43.236 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705676 2026-02-23T08:14:43.236 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:43.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705676 2026-02-23T08:14:43.238 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 38654705676 2026-02-23T08:14:43.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-23T08:14:43.238 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:14:43.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705676 -lt 38654705676 2026-02-23T08:14:43.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:43.467 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574859 2026-02-23T08:14:43.467 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:43.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:14:43.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574859 2026-02-23T08:14:43.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:43.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574859 2026-02-23T08:14:43.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574859' 2026-02-23T08:14:43.471 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 55834574859 2026-02-23T08:14:43.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:14:43.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574859 -lt 55834574859 2026-02-23T08:14:43.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:43.690 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-73014444043 2026-02-23T08:14:43.690 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:43.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:14:43.692 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-73014444043 2026-02-23T08:14:43.692 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:43.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444043 2026-02-23T08:14:43.694 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 73014444043' 2026-02-23T08:14:43.694 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 73014444043 2026-02-23T08:14:43.694 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:14:43.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444043 -lt 73014444043 2026-02-23T08:14:43.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:43.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280522 2026-02-23T08:14:43.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:43.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:14:43.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280522 2026-02-23T08:14:43.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:43.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280522 2026-02-23T08:14:43.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280522' 2026-02-23T08:14:43.927 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 94489280522 2026-02-23T08:14:43.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:14:44.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280522 -lt 94489280522 2026-02-23T08:14:44.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:44.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-115964117002 2026-02-23T08:14:44.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:44.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:14:44.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-115964117002 2026-02-23T08:14:44.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:44.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117002 2026-02-23T08:14:44.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 115964117002' 2026-02-23T08:14:44.148 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 115964117002 2026-02-23T08:14:44.149 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:14:44.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117002 -lt 115964117002 2026-02-23T08:14:44.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:44.379 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-133143986185 2026-02-23T08:14:44.379 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:44.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:14:44.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-133143986185 2026-02-23T08:14:44.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:44.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=133143986185 2026-02-23T08:14:44.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 133143986185' 2026-02-23T08:14:44.383 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 133143986185 2026-02-23T08:14:44.384 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:14:44.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 133143986183 -lt 133143986185 2026-02-23T08:14:44.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:14:45.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:14:45.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:14:45.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 133143986185 -lt 133143986185 2026-02-23T08:14:45.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:45.815 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822664 2026-02-23T08:14:45.816 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:45.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:14:45.818 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822664 2026-02-23T08:14:45.818 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:45.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822664 2026-02-23T08:14:45.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822664' 2026-02-23T08:14:45.819 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.7 seq 154618822664 2026-02-23T08:14:45.820 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:14:46.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822665 -lt 154618822664 2026-02-23T08:14:46.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:46.040 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659144 2026-02-23T08:14:46.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:46.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:14:46.042 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659144 2026-02-23T08:14:46.043 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:46.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659144 2026-02-23T08:14:46.044 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.8 seq 176093659144 2026-02-23T08:14:46.045 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659144' 2026-02-23T08:14:46.045 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:14:46.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659144 -lt 176093659144 2026-02-23T08:14:46.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:46.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-193273528327 2026-02-23T08:14:46.264 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:46.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:14:46.266 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-193273528327 2026-02-23T08:14:46.266 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:46.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=193273528327 2026-02-23T08:14:46.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 193273528327' 2026-02-23T08:14:46.267 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.9 seq 193273528327 2026-02-23T08:14:46.268 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:14:46.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 193273528327 -lt 193273528327 2026-02-23T08:14:46.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:46.484 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364806 2026-02-23T08:14:46.484 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:46.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:14:46.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364806 2026-02-23T08:14:46.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:46.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364806 2026-02-23T08:14:46.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364806' 2026-02-23T08:14:46.488 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.10 seq 214748364806 2026-02-23T08:14:46.488 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:14:46.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364807 -lt 214748364806 2026-02-23T08:14:46.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:14:46.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:14:46.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:14:47.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-23T08:14:47.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:14:47.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:14:47.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:14:47.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:14:47.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:14:47.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:14:47.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:14:47.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-23T08:14:47.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:14:47.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:14:47.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:14:47.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-23T08:14:47.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:14:47.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:14:47.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-23T08:14:47.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:TEST_chunk_mapping 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_put_get_isa 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_put_get_jerasure 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_put_get_lrc_advanced 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_put_get_lrc_kml 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_put_get_shec' 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: local payload=ABC 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:267: TEST_alignment_constraints: echo ABC 2026-02-23T08:14:47.516 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: chunk_size 2026-02-23T08:14:47.517 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-23T08:14:47.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-23T08:14:47.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-23T08:14:47.528 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-23T08:14:47.543 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-23T08:14:47.543 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-23T08:14:47.543 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-23T08:14:47.543 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-23T08:14:47.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: local stripe_unit=4096 2026-02-23T08:14:47.544 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-23T08:14:47.544 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: grep k= 2026-02-23T08:14:47.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: eval local k=2 2026-02-23T08:14:47.766 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: local k=2 2026-02-23T08:14:47.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:275: TEST_alignment_constraints: local block_size=8191 2026-02-23T08:14:47.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:276: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-23T08:14:47.767 INFO:tasks.workunit.client.0.trial186.stderr:2+0 records in 2026-02-23T08:14:47.768 INFO:tasks.workunit.client.0.trial186.stderr:2+0 records out 2026-02-23T08:14:47.768 INFO:tasks.workunit.client.0.trial186.stderr:16382 bytes (16 kB, 16 KiB) copied, 0.000106652 s, 154 MB/s 2026-02-23T08:14:47.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:277: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-23T08:14:47.790 INFO:tasks.workunit.client.0.trial186.stderr:INFO: op_size has been rounded to 8192 2026-02-23T08:14:47.794 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:279: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:14:47.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-23T08:14:47.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-23T08:14:47.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-23T08:14:47.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:320: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-23T08:14:47.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-23T08:14:47.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=ecpool 2026-02-23T08:14:47.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=0 2026-02-23T08:14:47.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=1 2026-02-23T08:14:47.796 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-23T08:14:47.796 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-23T08:14:47.806 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-23T08:14:47.807 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-23T08:14:47.807 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-23T08:14:47.821 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-23T08:14:47.821 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-23T08:14:47.821 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-23T08:14:47.821 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-23T08:14:47.821 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-23T08:14:47.821 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-23T08:14:47.822 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-23T08:14:47.831 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-23T08:14:47.831 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-23T08:14:47.831 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-23T08:14:47.843 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-23T08:14:47.843 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-23T08:14:47.843 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-23T08:14:47.843 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-23T08:14:47.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-23T08:14:47.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-23T08:14:47.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-23T08:14:47.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-23T08:14:47.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-23T08:14:47.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-23T08:14:47.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-23T08:14:47.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-23T08:14:47.898 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-23T08:14:47.898 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:14:48.115 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:14:48.115 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr:0' 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-23T08:14:48.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-23T08:14:48.179 INFO:tasks.workunit.client.0.trial186.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-23T08:14:48.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-23T08:14:48.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-23T08:14:48.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-23T08:14:48.256 INFO:tasks.workunit.client.0.trial186.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-23T08:14:48.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-23T08:14:48.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-23T08:14:48.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-23T08:14:48.328 INFO:tasks.workunit.client.0.trial186.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-23T08:14:48.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-23T08:14:48.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-23T08:14:48.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-23T08:14:48.398 INFO:tasks.workunit.client.0.trial186.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-23T08:14:48.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-23T08:14:48.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-23T08:14:48.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:14:48.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-23T08:14:48.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-23T08:14:48.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-23T08:14:48.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-23T08:14:48.404 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-23T08:14:48.404 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:14:48.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:14:48.618 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:14:48.618 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:14:48.618 INFO:tasks.workunit.client.0.trial186.stderr:0' 2026-02-23T08:14:48.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:14:48.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-23T08:14:48.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-23T08:14:48.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:14:48.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-23T08:14:48.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:14:48.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-23T08:14:48.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-23T08:14:48.621 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:14:48.621 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:14:48.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:14:48.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:14:48.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:14:48.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:14:48.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-23T08:14:48.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-23T08:14:48.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:14:48.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-23T08:14:48.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:14:48.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-23T08:14:48.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:49.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:49.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:start osd.3 2026-02-23T08:14:49.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:49.129 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-23T08:14:49.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-23T08:14:49.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:14:49.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:14:49.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:14:49.146 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:49.140+0000 7fdc9d865a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:49.147 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:49.144+0000 7fdc9d865a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:49.148 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:49.144+0000 7fdc9d865a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:49.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:49.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:14:49.504 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:49.500+0000 7fdc9d865a40 -1 Falling back to public interface 2026-02-23T08:14:49.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:50.244 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:50.240+0000 7fdc9d865a40 -1 osd.3 63 log_to_monitors true 2026-02-23T08:14:50.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:50.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:50.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:50.568 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:14:50.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:50.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:14:50.779 INFO:tasks.workunit.client.0.trial186.stderr:osd.3 up in weight 1 up_from 67 up_thru 67 down_at 64 last_clean_interval [17,63) [v2:127.0.0.1:6826/137748302,v1:127.0.0.1:6827/137748302] [v2:127.0.0.1:6828/137748302,v1:127.0.0.1:6829/137748302] exists,up f8017f9d-8067-41af-8023-ca42895cb644 2026-02-23T08:14:50.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:50.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:50.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:50.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:14:50.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:14:50.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:14:50.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:14:50.780 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:14:50.781 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:14:50.781 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:14:50.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:14:50.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:14:50.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:14:50.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:14:50.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:14:50.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:14:50.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:14:50.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:14:50.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:14:51.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:14:51.099 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:14:51.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:14:51.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:14:51.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836497 2026-02-23T08:14:51.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836497 2026-02-23T08:14:51.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497' 2026-02-23T08:14:51.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:14:51.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705680 2026-02-23T08:14:51.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705680 2026-02-23T08:14:51.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680' 2026-02-23T08:14:51.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.256 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:14:51.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-23T08:14:51.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-23T08:14:51.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863' 2026-02-23T08:14:51.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:14:51.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808835 2026-02-23T08:14:51.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808835 2026-02-23T08:14:51.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835' 2026-02-23T08:14:51.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:14:51.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280526 2026-02-23T08:14:51.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280526 2026-02-23T08:14:51.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835 4-94489280526' 2026-02-23T08:14:51.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.495 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:14:51.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117005 2026-02-23T08:14:51.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117005 2026-02-23T08:14:51.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835 4-94489280526 5-115964117005' 2026-02-23T08:14:51.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.572 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:14:51.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986189 2026-02-23T08:14:51.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986189 2026-02-23T08:14:51.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835 4-94489280526 5-115964117005 6-133143986189' 2026-02-23T08:14:51.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.653 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:14:51.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822668 2026-02-23T08:14:51.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822668 2026-02-23T08:14:51.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835 4-94489280526 5-115964117005 6-133143986189 7-154618822668' 2026-02-23T08:14:51.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:14:51.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659148 2026-02-23T08:14:51.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659148 2026-02-23T08:14:51.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835 4-94489280526 5-115964117005 6-133143986189 7-154618822668 8-176093659148' 2026-02-23T08:14:51.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.809 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:14:51.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528331 2026-02-23T08:14:51.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528331 2026-02-23T08:14:51.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835 4-94489280526 5-115964117005 6-133143986189 7-154618822668 8-176093659148 9-193273528331' 2026-02-23T08:14:51.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:51.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:14:51.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364810 2026-02-23T08:14:51.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364810 2026-02-23T08:14:51.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863 3-287762808835 4-94489280526 5-115964117005 6-133143986189 7-154618822668 8-176093659148 9-193273528331 10-214748364810' 2026-02-23T08:14:51.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:51.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836497 2026-02-23T08:14:51.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:51.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:14:51.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836497 2026-02-23T08:14:51.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:51.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836497 2026-02-23T08:14:51.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-23T08:14:51.970 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 21474836497 2026-02-23T08:14:51.971 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:14:52.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836495 -lt 21474836497 2026-02-23T08:14:52.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:14:53.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:14:53.189 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:14:53.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-23T08:14:53.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:53.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705680 2026-02-23T08:14:53.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:53.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:14:53.421 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705680 2026-02-23T08:14:53.421 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:53.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705680 2026-02-23T08:14:53.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-23T08:14:53.422 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 38654705680 2026-02-23T08:14:53.423 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:14:53.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-23T08:14:53.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:53.651 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574863 2026-02-23T08:14:53.651 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:53.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:14:53.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574863 2026-02-23T08:14:53.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:53.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574863 2026-02-23T08:14:53.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-23T08:14:53.654 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 55834574863 2026-02-23T08:14:53.654 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:14:53.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-23T08:14:53.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:53.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-287762808835 2026-02-23T08:14:53.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:53.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:14:53.874 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808835 2026-02-23T08:14:53.874 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:53.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808835 2026-02-23T08:14:53.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 287762808835' 2026-02-23T08:14:53.876 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 287762808835 2026-02-23T08:14:53.876 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:14:54.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808835 -lt 287762808835 2026-02-23T08:14:54.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:54.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280526 2026-02-23T08:14:54.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:54.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:14:54.099 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280526 2026-02-23T08:14:54.099 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:54.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280526 2026-02-23T08:14:54.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280526' 2026-02-23T08:14:54.100 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.4 seq 94489280526 2026-02-23T08:14:54.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:14:54.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280526 -lt 94489280526 2026-02-23T08:14:54.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:54.319 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-115964117005 2026-02-23T08:14:54.319 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:54.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:14:54.321 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-115964117005 2026-02-23T08:14:54.321 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:54.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117005 2026-02-23T08:14:54.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 115964117005' 2026-02-23T08:14:54.322 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.5 seq 115964117005 2026-02-23T08:14:54.323 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:14:54.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117006 -lt 115964117005 2026-02-23T08:14:54.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:54.540 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-133143986189 2026-02-23T08:14:54.540 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:54.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:14:54.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-133143986189 2026-02-23T08:14:54.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:54.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=133143986189 2026-02-23T08:14:54.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 133143986189' 2026-02-23T08:14:54.543 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.6 seq 133143986189 2026-02-23T08:14:54.544 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:14:54.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 133143986189 -lt 133143986189 2026-02-23T08:14:54.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:54.767 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822668 2026-02-23T08:14:54.767 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:54.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:14:54.769 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822668 2026-02-23T08:14:54.769 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:54.770 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822668 2026-02-23T08:14:54.770 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822668' 2026-02-23T08:14:54.770 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.7 seq 154618822668 2026-02-23T08:14:54.771 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:14:54.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822669 -lt 154618822668 2026-02-23T08:14:54.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:54.989 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659148 2026-02-23T08:14:54.989 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:54.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:14:54.991 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659148 2026-02-23T08:14:54.991 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:54.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659148 2026-02-23T08:14:54.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659148' 2026-02-23T08:14:54.993 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.8 seq 176093659148 2026-02-23T08:14:54.993 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:14:55.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659148 -lt 176093659148 2026-02-23T08:14:55.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:55.220 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-193273528331 2026-02-23T08:14:55.220 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:55.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:14:55.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-193273528331 2026-02-23T08:14:55.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:55.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=193273528331 2026-02-23T08:14:55.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 193273528331' 2026-02-23T08:14:55.223 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.9 seq 193273528331 2026-02-23T08:14:55.223 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:14:55.450 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 193273528331 -lt 193273528331 2026-02-23T08:14:55.450 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:55.451 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364810 2026-02-23T08:14:55.451 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:55.452 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:14:55.453 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364810 2026-02-23T08:14:55.453 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:55.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364810 2026-02-23T08:14:55.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364810' 2026-02-23T08:14:55.454 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.10 seq 214748364810 2026-02-23T08:14:55.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:14:55.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364811 -lt 214748364810 2026-02-23T08:14:55.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:14:55.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:14:55.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:14:55.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-23T08:14:55.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:14:55.952 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:14:55.952 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:14:55.952 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:14:55.952 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:14:55.952 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:14:55.952 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:14:56.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-23T08:14:56.162 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:14:56.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:14:56.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:14:56.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-23T08:14:56.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:14:56.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-23T08:14:56.459 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:14:56.460 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:14:56.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:14:56.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:14:56.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:14:56.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:14:56.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-23T08:14:56.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-23T08:14:56.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:14:56.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-23T08:14:56.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:14:56.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-23T08:14:56.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-23T08:14:56.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-23T08:14:56.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-23T08:14:56.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:14:56.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-23T08:14:56.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:14:56.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-23T08:14:56.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:14:56.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:14:56.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:14:56.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:14:56.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-23T08:14:56.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-23T08:14:56.969 INFO:tasks.workunit.client.0.trial186.stderr:start osd.5 2026-02-23T08:14:56.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:14:56.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-23T08:14:56.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-23T08:14:56.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:14:56.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:14:56.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:14:56.985 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:56.980+0000 7f4e2ec20a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:56.986 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:56.980+0000 7f4e2ec20a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:56.987 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:56.984+0000 7f4e2ec20a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:14:57.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:57.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:14:57.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:14:57.448 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:57.444+0000 7f4e2ec20a40 -1 Falling back to public interface 2026-02-23T08:14:58.060 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:14:58.056+0000 7f4e2ec20a40 -1 osd.5 69 log_to_monitors true 2026-02-23T08:14:58.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:14:58.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:14:58.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:14:58.402 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:14:58.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:14:58.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:14:58.617 INFO:tasks.workunit.client.0.trial186.stderr:osd.5 up in weight 1 up_from 72 up_thru 72 down_at 69 last_clean_interval [27,69) [v2:127.0.0.1:6842/1862211913,v1:127.0.0.1:6843/1862211913] [v2:127.0.0.1:6844/1862211913,v1:127.0.0.1:6845/1862211913] exists,up 060155e1-01a7-4a6c-87d7-b2b4f55eaf2f 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:14:58.618 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:14:58.619 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:14:58.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:14:58.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:14:58.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:14:58.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:14:58.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:14:58.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:14:58.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:14:58.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:14:58.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:14:58.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:14:58.942 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:14:58.942 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:14:58.942 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:14:58.942 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:58.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:14:59.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836500 2026-02-23T08:14:59.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836500 2026-02-23T08:14:59.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500' 2026-02-23T08:14:59.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.025 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:14:59.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705684 2026-02-23T08:14:59.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705684 2026-02-23T08:14:59.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684' 2026-02-23T08:14:59.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.107 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:14:59.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574867 2026-02-23T08:14:59.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574867 2026-02-23T08:14:59.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867' 2026-02-23T08:14:59.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:14:59.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808838 2026-02-23T08:14:59.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808838 2026-02-23T08:14:59.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838' 2026-02-23T08:14:59.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.269 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:14:59.347 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280529 2026-02-23T08:14:59.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280529 2026-02-23T08:14:59.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838 4-94489280529' 2026-02-23T08:14:59.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.348 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:14:59.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645315 2026-02-23T08:14:59.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645315 2026-02-23T08:14:59.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838 4-94489280529 5-309237645315' 2026-02-23T08:14:59.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.429 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:14:59.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986192 2026-02-23T08:14:59.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986192 2026-02-23T08:14:59.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838 4-94489280529 5-309237645315 6-133143986192' 2026-02-23T08:14:59.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.507 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:14:59.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822672 2026-02-23T08:14:59.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822672 2026-02-23T08:14:59.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838 4-94489280529 5-309237645315 6-133143986192 7-154618822672' 2026-02-23T08:14:59.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:14:59.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659151 2026-02-23T08:14:59.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659151 2026-02-23T08:14:59.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838 4-94489280529 5-309237645315 6-133143986192 7-154618822672 8-176093659151' 2026-02-23T08:14:59.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:14:59.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528334 2026-02-23T08:14:59.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528334 2026-02-23T08:14:59.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838 4-94489280529 5-309237645315 6-133143986192 7-154618822672 8-176093659151 9-193273528334' 2026-02-23T08:14:59.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:14:59.747 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:14:59.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364814 2026-02-23T08:14:59.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364814 2026-02-23T08:14:59.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836500 1-38654705684 2-55834574867 3-287762808838 4-94489280529 5-309237645315 6-133143986192 7-154618822672 8-176093659151 9-193273528334 10-214748364814' 2026-02-23T08:14:59.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:14:59.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836500 2026-02-23T08:14:59.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:14:59.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:14:59.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836500 2026-02-23T08:14:59.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:14:59.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836500 2026-02-23T08:14:59.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-23T08:14:59.830 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 21474836500 2026-02-23T08:14:59.830 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:00.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836498 -lt 21474836500 2026-02-23T08:15:00.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:15:01.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:15:01.057 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:01.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836501 -lt 21474836500 2026-02-23T08:15:01.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:01.269 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705684 2026-02-23T08:15:01.269 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:01.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:15:01.270 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705684 2026-02-23T08:15:01.270 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:01.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705684 2026-02-23T08:15:01.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705684' 2026-02-23T08:15:01.272 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 38654705684 2026-02-23T08:15:01.273 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:15:01.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705684 -lt 38654705684 2026-02-23T08:15:01.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:01.494 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574867 2026-02-23T08:15:01.495 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:01.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:15:01.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574867 2026-02-23T08:15:01.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:01.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574867 2026-02-23T08:15:01.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574867' 2026-02-23T08:15:01.498 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 55834574867 2026-02-23T08:15:01.498 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:15:01.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574867 -lt 55834574867 2026-02-23T08:15:01.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:01.723 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-287762808838 2026-02-23T08:15:01.723 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:01.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:15:01.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808838 2026-02-23T08:15:01.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:01.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808838 2026-02-23T08:15:01.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 287762808838' 2026-02-23T08:15:01.726 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 287762808838 2026-02-23T08:15:01.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:15:01.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808839 -lt 287762808838 2026-02-23T08:15:01.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:01.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280529 2026-02-23T08:15:01.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:01.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:15:01.946 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280529 2026-02-23T08:15:01.946 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:01.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280529 2026-02-23T08:15:01.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280529' 2026-02-23T08:15:01.947 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.4 seq 94489280529 2026-02-23T08:15:01.947 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:15:02.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280530 -lt 94489280529 2026-02-23T08:15:02.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:02.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-309237645315 2026-02-23T08:15:02.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:02.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:15:02.177 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645315 2026-02-23T08:15:02.177 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:02.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=309237645315 2026-02-23T08:15:02.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 309237645315' 2026-02-23T08:15:02.179 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.5 seq 309237645315 2026-02-23T08:15:02.179 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:15:02.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 309237645315 -lt 309237645315 2026-02-23T08:15:02.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:02.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-133143986192 2026-02-23T08:15:02.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:02.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:15:02.396 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-133143986192 2026-02-23T08:15:02.396 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:02.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=133143986192 2026-02-23T08:15:02.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 133143986192' 2026-02-23T08:15:02.399 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.6 seq 133143986192 2026-02-23T08:15:02.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:15:02.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 133143986193 -lt 133143986192 2026-02-23T08:15:02.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:02.621 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822672 2026-02-23T08:15:02.621 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:02.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:15:02.623 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822672 2026-02-23T08:15:02.623 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:02.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822672 2026-02-23T08:15:02.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822672' 2026-02-23T08:15:02.624 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.7 seq 154618822672 2026-02-23T08:15:02.624 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:15:02.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822672 -lt 154618822672 2026-02-23T08:15:02.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:02.851 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659151 2026-02-23T08:15:02.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:02.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:15:02.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659151 2026-02-23T08:15:02.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:02.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659151 2026-02-23T08:15:02.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659151' 2026-02-23T08:15:02.854 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.8 seq 176093659151 2026-02-23T08:15:02.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:15:03.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659152 -lt 176093659151 2026-02-23T08:15:03.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:03.067 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-193273528334 2026-02-23T08:15:03.068 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:03.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:15:03.069 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-193273528334 2026-02-23T08:15:03.069 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:03.070 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=193273528334 2026-02-23T08:15:03.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 193273528334' 2026-02-23T08:15:03.071 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.9 seq 193273528334 2026-02-23T08:15:03.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:15:03.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 193273528335 -lt 193273528334 2026-02-23T08:15:03.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:03.293 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364814 2026-02-23T08:15:03.293 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:03.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:15:03.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364814 2026-02-23T08:15:03.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:03.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364814 2026-02-23T08:15:03.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364814' 2026-02-23T08:15:03.296 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.10 seq 214748364814 2026-02-23T08:15:03.296 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:15:03.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364814 -lt 214748364814 2026-02-23T08:15:03.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:15:03.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:03.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:15:03.808 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:15:04.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-23T08:15:04.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:15:04.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:04.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:04.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-23T08:15:04.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:15:04.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:15:04.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:322: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-23T08:15:04.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-23T08:15:04.822 INFO:tasks.workunit.client.0.trial186.stdout:crush-device-class= 2026-02-23T08:15:04.822 INFO:tasks.workunit.client.0.trial186.stdout:crush-failure-domain=host 2026-02-23T08:15:04.822 INFO:tasks.workunit.client.0.trial186.stdout:crush-num-failure-domains=0 2026-02-23T08:15:04.822 INFO:tasks.workunit.client.0.trial186.stdout:crush-osds-per-failure-domain=0 2026-02-23T08:15:04.822 INFO:tasks.workunit.client.0.trial186.stdout:crush-root=default 2026-02-23T08:15:04.823 INFO:tasks.workunit.client.0.trial186.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-23T08:15:04.823 INFO:tasks.workunit.client.0.trial186.stdout:k=-1 2026-02-23T08:15:04.823 INFO:tasks.workunit.client.0.trial186.stdout:l=-1 2026-02-23T08:15:04.823 INFO:tasks.workunit.client.0.trial186.stdout:layers=[ [ "cDD", "" ] ] 2026-02-23T08:15:04.823 INFO:tasks.workunit.client.0.trial186.stdout:m=-1 2026-02-23T08:15:04.823 INFO:tasks.workunit.client.0.trial186.stdout:mapping=_DD 2026-02-23T08:15:04.823 INFO:tasks.workunit.client.0.trial186.stdout:plugin=lrc 2026-02-23T08:15:04.833 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:328: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-23T08:15:04.833 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-23T08:15:05.175 INFO:tasks.workunit.client.0.trial186.stderr:pool 'remap-pool' already exists 2026-02-23T08:15:05.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:15:06.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:336: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-23T08:15:06.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-23T08:15:06.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=remap-pool 2026-02-23T08:15:06.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=1 2026-02-23T08:15:06.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=2 2026-02-23T08:15:06.188 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-23T08:15:06.188 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-23T08:15:06.199 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-23T08:15:06.199 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-23T08:15:06.200 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-23T08:15:06.214 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-23T08:15:06.214 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-23T08:15:06.214 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-23T08:15:06.214 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-23T08:15:06.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-23T08:15:06.214 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-23T08:15:06.215 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-23T08:15:06.223 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-23T08:15:06.224 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-23T08:15:06.224 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-23T08:15:06.237 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-23T08:15:06.237 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-23T08:15:06.237 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-23T08:15:06.238 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-23T08:15:06.238 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-23T08:15:06.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-23T08:15:06.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-23T08:15:06.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-23T08:15:06.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-23T08:15:06.283 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-23T08:15:06.283 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-23T08:15:06.283 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-23T08:15:06.284 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-23T08:15:06.284 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:06.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-23T08:15:06.497 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:15:06.497 INFO:tasks.workunit.client.0.trial186.stderr:9' 2026-02-23T08:15:06.497 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-23T08:15:06.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-23T08:15:06.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-23T08:15:06.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-23T08:15:06.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-23T08:15:06.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-23T08:15:06.561 INFO:tasks.workunit.client.0.trial186.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-23T08:15:06.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-23T08:15:06.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-23T08:15:06.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-23T08:15:06.636 INFO:tasks.workunit.client.0.trial186.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-23T08:15:06.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-23T08:15:06.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-23T08:15:06.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-23T08:15:06.710 INFO:tasks.workunit.client.0.trial186.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-23T08:15:06.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-23T08:15:06.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-23T08:15:06.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:06.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-23T08:15:06.714 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-23T08:15:06.714 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-23T08:15:06.714 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-23T08:15:06.715 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-23T08:15:06.715 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:06.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-23T08:15:06.940 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:15:06.940 INFO:tasks.workunit.client.0.trial186.stderr:9' 2026-02-23T08:15:06.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-23T08:15:06.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-23T08:15:06.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:15:06.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-23T08:15:06.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:15:06.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:15:06.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:15:06.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:15:06.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:15:07.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:15:07.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:07.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-23T08:15:07.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:15:07.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-23T08:15:07.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:15:07.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-23T08:15:07.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:07.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-23T08:15:07.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:15:07.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:15:07.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-23T08:15:07.456 INFO:tasks.workunit.client.0.trial186.stderr:start osd.6 2026-02-23T08:15:07.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:15:07.457 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-23T08:15:07.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-23T08:15:07.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:15:07.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:15:07.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:15:07.471 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:07.468+0000 7f1d829cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:15:07.472 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:07.468+0000 7f1d829cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:15:07.473 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:07.468+0000 7f1d829cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:15:07.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-23T08:15:07.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:15:07.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-23T08:15:07.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:15:07.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:15:07.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:15:07.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:15:07.677 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:15:07.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:15:07.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:15:07.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:15:07.915 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:07.912+0000 7f1d829cfa40 -1 Falling back to public interface 2026-02-23T08:15:08.571 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:08.568+0000 7f1d829cfa40 -1 osd.6 81 log_to_monitors true 2026-02-23T08:15:08.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:15:08.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:15:08.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:15:08.896 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:08.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:15:08.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:15:09.113 INFO:tasks.workunit.client.0.trial186.stderr:osd.6 up in weight 1 up_from 85 up_thru 85 down_at 82 last_clean_interval [31,81) [v2:127.0.0.1:6850/803587027,v1:127.0.0.1:6851/803587027] [v2:127.0.0.1:6852/803587027,v1:127.0.0.1:6853/803587027] exists,up 36ce2271-f4a8-41ef-8ec1-17c1156151c9 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:15:09.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:15:09.115 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:15:09.116 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:15:09.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:15:09.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:15:09.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:15:09.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:15:09.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:15:09.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:15:09.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:15:09.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:15:09.221 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:15:09.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:15:09.430 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:09.430 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.431 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:15:09.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836504 2026-02-23T08:15:09.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836504 2026-02-23T08:15:09.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504' 2026-02-23T08:15:09.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.512 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:15:09.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705688 2026-02-23T08:15:09.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705688 2026-02-23T08:15:09.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688' 2026-02-23T08:15:09.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.591 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:15:09.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574871 2026-02-23T08:15:09.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574871 2026-02-23T08:15:09.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871' 2026-02-23T08:15:09.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:15:09.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808842 2026-02-23T08:15:09.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808842 2026-02-23T08:15:09.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842' 2026-02-23T08:15:09.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.750 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:15:09.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280534 2026-02-23T08:15:09.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280534 2026-02-23T08:15:09.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842 4-94489280534' 2026-02-23T08:15:09.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.831 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:15:09.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645319 2026-02-23T08:15:09.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645319 2026-02-23T08:15:09.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842 4-94489280534 5-309237645319' 2026-02-23T08:15:09.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.913 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:15:09.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220163 2026-02-23T08:15:09.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220163 2026-02-23T08:15:09.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842 4-94489280534 5-309237645319 6-365072220163' 2026-02-23T08:15:09.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:09.992 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:15:10.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822676 2026-02-23T08:15:10.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822676 2026-02-23T08:15:10.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842 4-94489280534 5-309237645319 6-365072220163 7-154618822676' 2026-02-23T08:15:10.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:10.068 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:15:10.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659155 2026-02-23T08:15:10.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659155 2026-02-23T08:15:10.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842 4-94489280534 5-309237645319 6-365072220163 7-154618822676 8-176093659155' 2026-02-23T08:15:10.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:10.148 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:15:10.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528338 2026-02-23T08:15:10.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528338 2026-02-23T08:15:10.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842 4-94489280534 5-309237645319 6-365072220163 7-154618822676 8-176093659155 9-193273528338' 2026-02-23T08:15:10.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:10.227 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:15:10.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364818 2026-02-23T08:15:10.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364818 2026-02-23T08:15:10.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836504 1-38654705688 2-55834574871 3-287762808842 4-94489280534 5-309237645319 6-365072220163 7-154618822676 8-176093659155 9-193273528338 10-214748364818' 2026-02-23T08:15:10.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:10.309 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836504 2026-02-23T08:15:10.309 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:10.310 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:15:10.310 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836504 2026-02-23T08:15:10.310 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:10.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836504 2026-02-23T08:15:10.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-23T08:15:10.311 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 21474836504 2026-02-23T08:15:10.311 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:10.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836502 -lt 21474836504 2026-02-23T08:15:10.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:15:11.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:15:11.525 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:11.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836505 -lt 21474836504 2026-02-23T08:15:11.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:11.744 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705688 2026-02-23T08:15:11.744 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:11.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:15:11.747 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705688 2026-02-23T08:15:11.747 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:11.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705688 2026-02-23T08:15:11.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705688' 2026-02-23T08:15:11.749 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 38654705688 2026-02-23T08:15:11.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:15:11.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705688 -lt 38654705688 2026-02-23T08:15:11.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:11.976 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574871 2026-02-23T08:15:11.976 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:11.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:15:11.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574871 2026-02-23T08:15:11.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:11.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574871 2026-02-23T08:15:11.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574871' 2026-02-23T08:15:11.979 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 55834574871 2026-02-23T08:15:11.979 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:15:12.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574871 -lt 55834574871 2026-02-23T08:15:12.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:12.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-287762808842 2026-02-23T08:15:12.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:12.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:15:12.199 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808842 2026-02-23T08:15:12.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:12.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808842 2026-02-23T08:15:12.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 287762808842' 2026-02-23T08:15:12.201 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 287762808842 2026-02-23T08:15:12.201 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:15:12.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808843 -lt 287762808842 2026-02-23T08:15:12.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:12.429 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280534 2026-02-23T08:15:12.429 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:12.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:15:12.430 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280534 2026-02-23T08:15:12.430 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:12.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280534 2026-02-23T08:15:12.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280534' 2026-02-23T08:15:12.432 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.4 seq 94489280534 2026-02-23T08:15:12.432 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:15:12.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280534 -lt 94489280534 2026-02-23T08:15:12.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:12.651 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-309237645319 2026-02-23T08:15:12.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:12.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:15:12.653 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645319 2026-02-23T08:15:12.653 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:12.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=309237645319 2026-02-23T08:15:12.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 309237645319' 2026-02-23T08:15:12.655 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.5 seq 309237645319 2026-02-23T08:15:12.655 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:15:12.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 309237645319 -lt 309237645319 2026-02-23T08:15:12.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:12.871 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-365072220163 2026-02-23T08:15:12.871 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:12.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:15:12.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-365072220163 2026-02-23T08:15:12.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:12.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220163 2026-02-23T08:15:12.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 365072220163' 2026-02-23T08:15:12.874 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.6 seq 365072220163 2026-02-23T08:15:12.874 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:15:13.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220163 -lt 365072220163 2026-02-23T08:15:13.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:13.099 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822676 2026-02-23T08:15:13.099 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:13.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:15:13.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822676 2026-02-23T08:15:13.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:13.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822676 2026-02-23T08:15:13.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822676' 2026-02-23T08:15:13.102 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.7 seq 154618822676 2026-02-23T08:15:13.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:15:13.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822676 -lt 154618822676 2026-02-23T08:15:13.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:13.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659155 2026-02-23T08:15:13.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:13.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:15:13.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659155 2026-02-23T08:15:13.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:13.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659155 2026-02-23T08:15:13.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659155' 2026-02-23T08:15:13.335 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.8 seq 176093659155 2026-02-23T08:15:13.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:15:13.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659156 -lt 176093659155 2026-02-23T08:15:13.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:13.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-193273528338 2026-02-23T08:15:13.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:13.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:15:13.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-193273528338 2026-02-23T08:15:13.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:13.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=193273528338 2026-02-23T08:15:13.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 193273528338' 2026-02-23T08:15:13.558 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.9 seq 193273528338 2026-02-23T08:15:13.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:15:13.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 193273528339 -lt 193273528338 2026-02-23T08:15:13.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:13.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364818 2026-02-23T08:15:13.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:13.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:15:13.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364818 2026-02-23T08:15:13.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:13.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364818 2026-02-23T08:15:13.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364818' 2026-02-23T08:15:13.787 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.10 seq 214748364818 2026-02-23T08:15:13.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:15:14.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364818 -lt 214748364818 2026-02-23T08:15:14.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:15:14.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:14.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:14.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-23T08:15:14.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:15:14.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:15:14.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:15:14.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:15:14.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:15:14.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:15:14.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:15:14.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-23T08:15:14.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:15:14.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:14.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:14.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-23T08:15:14.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:15:14.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:15:14.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-23T08:15:14.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:15:14.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:15:15.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:15:15.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:15.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-23T08:15:15.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:15:15.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-23T08:15:15.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:15:15.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-23T08:15:15.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:15:15.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:15:15.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:15:15.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-23T08:15:15.261 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-23T08:15:15.261 INFO:tasks.workunit.client.0.trial186.stderr:start osd.9 2026-02-23T08:15:15.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=c25d1837-b1f1-4bc4-82c4-3faa18e5f85a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.46012/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:15:15.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-23T08:15:15.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-23T08:15:15.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:15:15.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:15:15.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:15:15.285 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:15.280+0000 7fd9ff3f5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:15:15.287 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:15.284+0000 7fd9ff3f5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:15:15.288 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:15.284+0000 7fd9ff3f5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:15:15.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-23T08:15:15.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:15:15.775 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:15.772+0000 7fd9ff3f5a40 -1 Falling back to public interface 2026-02-23T08:15:16.451 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:15:16.448+0000 7fd9ff3f5a40 -1 osd.9 87 log_to_monitors true 2026-02-23T08:15:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:15:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:15:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:15:16.704 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:15:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-23T08:15:16.920 INFO:tasks.workunit.client.0.trial186.stderr:osd.9 up in weight 1 up_from 90 up_thru 90 down_at 87 last_clean_interval [45,87) [v2:127.0.0.1:6874/383726888,v1:127.0.0.1:6875/383726888] [v2:127.0.0.1:6876/383726888,v1:127.0.0.1:6877/383726888] exists,up 4472fe7f-4301-4f30-887f-b0cbf5ab6b82 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:15:16.921 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:15:16.922 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:15:16.922 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:15:16.922 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:15:16.922 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:15:17.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:15:17.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:15:17.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:15:17.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:15:17.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:15:17.012 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.224 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:15:17.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836508 2026-02-23T08:15:17.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836508 2026-02-23T08:15:17.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508' 2026-02-23T08:15:17.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:15:17.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705691 2026-02-23T08:15:17.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705691 2026-02-23T08:15:17.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691' 2026-02-23T08:15:17.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.382 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:15:17.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574874 2026-02-23T08:15:17.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574874 2026-02-23T08:15:17.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874' 2026-02-23T08:15:17.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.457 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:15:17.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808846 2026-02-23T08:15:17.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808846 2026-02-23T08:15:17.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846' 2026-02-23T08:15:17.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.538 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:15:17.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280537 2026-02-23T08:15:17.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280537 2026-02-23T08:15:17.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846 4-94489280537' 2026-02-23T08:15:17.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.622 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:15:17.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645322 2026-02-23T08:15:17.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645322 2026-02-23T08:15:17.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846 4-94489280537 5-309237645322' 2026-02-23T08:15:17.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.702 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:15:17.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220166 2026-02-23T08:15:17.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220166 2026-02-23T08:15:17.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846 4-94489280537 5-309237645322 6-365072220166' 2026-02-23T08:15:17.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.782 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:15:17.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822679 2026-02-23T08:15:17.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822679 2026-02-23T08:15:17.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846 4-94489280537 5-309237645322 6-365072220166 7-154618822679' 2026-02-23T08:15:17.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.864 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:15:17.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659159 2026-02-23T08:15:17.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659159 2026-02-23T08:15:17.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846 4-94489280537 5-309237645322 6-365072220166 7-154618822679 8-176093659159' 2026-02-23T08:15:17.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:17.945 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:15:18.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056643 2026-02-23T08:15:18.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056643 2026-02-23T08:15:18.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846 4-94489280537 5-309237645322 6-365072220166 7-154618822679 8-176093659159 9-386547056643' 2026-02-23T08:15:18.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:18.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:15:18.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364821 2026-02-23T08:15:18.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364821 2026-02-23T08:15:18.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574874 3-287762808846 4-94489280537 5-309237645322 6-365072220166 7-154618822679 8-176093659159 9-386547056643 10-214748364821' 2026-02-23T08:15:18.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:18.107 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836508 2026-02-23T08:15:18.107 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:18.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:15:18.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836508 2026-02-23T08:15:18.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:18.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836508 2026-02-23T08:15:18.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-23T08:15:18.109 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 21474836508 2026-02-23T08:15:18.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:18.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836506 -lt 21474836508 2026-02-23T08:15:18.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:15:19.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:15:19.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:19.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-23T08:15:19.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:19.554 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705691 2026-02-23T08:15:19.554 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:19.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:15:19.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705691 2026-02-23T08:15:19.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:19.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705691 2026-02-23T08:15:19.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705691' 2026-02-23T08:15:19.557 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 38654705691 2026-02-23T08:15:19.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:15:19.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705691 -lt 38654705691 2026-02-23T08:15:19.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:19.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574874 2026-02-23T08:15:19.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:19.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:15:19.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574874 2026-02-23T08:15:19.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:19.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574874 2026-02-23T08:15:19.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574874' 2026-02-23T08:15:19.788 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 55834574874 2026-02-23T08:15:19.789 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:15:20.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574875 -lt 55834574874 2026-02-23T08:15:20.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:20.013 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-287762808846 2026-02-23T08:15:20.013 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:20.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:15:20.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808846 2026-02-23T08:15:20.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:20.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808846 2026-02-23T08:15:20.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 287762808846' 2026-02-23T08:15:20.017 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 287762808846 2026-02-23T08:15:20.017 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:15:20.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808846 -lt 287762808846 2026-02-23T08:15:20.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:20.239 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280537 2026-02-23T08:15:20.239 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:20.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:15:20.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280537 2026-02-23T08:15:20.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:20.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280537 2026-02-23T08:15:20.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280537' 2026-02-23T08:15:20.242 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.4 seq 94489280537 2026-02-23T08:15:20.242 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:15:20.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280537 -lt 94489280537 2026-02-23T08:15:20.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:20.468 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-309237645322 2026-02-23T08:15:20.468 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:20.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:15:20.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645322 2026-02-23T08:15:20.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:20.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=309237645322 2026-02-23T08:15:20.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 309237645322' 2026-02-23T08:15:20.471 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.5 seq 309237645322 2026-02-23T08:15:20.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:15:20.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 309237645323 -lt 309237645322 2026-02-23T08:15:20.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:20.694 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-365072220166 2026-02-23T08:15:20.694 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:20.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:15:20.695 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-365072220166 2026-02-23T08:15:20.696 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:20.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220166 2026-02-23T08:15:20.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 365072220166' 2026-02-23T08:15:20.697 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.6 seq 365072220166 2026-02-23T08:15:20.697 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:15:20.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220167 -lt 365072220166 2026-02-23T08:15:20.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:20.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822679 2026-02-23T08:15:20.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:20.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:15:20.914 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822679 2026-02-23T08:15:20.914 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:20.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822679 2026-02-23T08:15:20.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822679' 2026-02-23T08:15:20.915 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.7 seq 154618822679 2026-02-23T08:15:20.916 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:15:21.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822680 -lt 154618822679 2026-02-23T08:15:21.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:21.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659159 2026-02-23T08:15:21.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:21.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:15:21.146 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659159 2026-02-23T08:15:21.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:21.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659159 2026-02-23T08:15:21.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659159' 2026-02-23T08:15:21.148 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.8 seq 176093659159 2026-02-23T08:15:21.149 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:15:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659159 -lt 176093659159 2026-02-23T08:15:21.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:21.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-386547056643 2026-02-23T08:15:21.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:21.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:15:21.375 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-386547056643 2026-02-23T08:15:21.375 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:21.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=386547056643 2026-02-23T08:15:21.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 386547056643' 2026-02-23T08:15:21.377 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.9 seq 386547056643 2026-02-23T08:15:21.377 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:15:21.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-23T08:15:21.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:21.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364821 2026-02-23T08:15:21.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:21.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:15:21.602 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364821 2026-02-23T08:15:21.602 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:21.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364821 2026-02-23T08:15:21.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364821' 2026-02-23T08:15:21.604 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.10 seq 214748364821 2026-02-23T08:15:21.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:15:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364822 -lt 214748364821 2026-02-23T08:15:21.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:15:21.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:21.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:22.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-23T08:15:22.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:15:22.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:15:22.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:15:22.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:15:22.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:15:22.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:15:22.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:15:22.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-23T08:15:22.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:15:22.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:22.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:22.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-23T08:15:22.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:15:22.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:15:22.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:338: TEST_chunk_mapping: delete_pool remap-pool 2026-02-23T08:15:22.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-23T08:15:22.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-23T08:15:22.879 INFO:tasks.workunit.client.0.trial186.stderr:pool 'remap-pool' does not exist 2026-02-23T08:15:22.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:339: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-23T08:15:23.170 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile remap-profile does not exist 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=isa 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-23T08:15:23.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-23T08:15:23.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-23T08:15:23.183 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-23T08:15:23.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-23T08:15:23.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-23T08:15:23.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-23T08:15:23.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-23T08:15:23.729 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-23T08:15:23.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-23T08:15:23.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-23T08:15:23.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-23T08:15:23.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-23T08:15:24.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-23T08:15:24.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-23T08:15:24.405 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-isa' already exists 2026-02-23T08:15:24.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:15:25.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-23T08:15:25.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-23T08:15:25.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-23T08:15:25.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:25.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:25.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-23T08:15:25.459 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-23T08:15:25.459 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-23T08:15:25.459 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:25.460 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-23T08:15:25.460 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:25.682 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr:10 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr:7' 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-23T08:15:25.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-23T08:15:25.931 INFO:tasks.workunit.client.0.trial186.stderr:osd.7 is already out. 2026-02-23T08:15:25.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-23T08:15:30.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-23T08:15:30.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-23T08:15:30.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:30.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-23T08:15:30.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-23T08:15:30.944 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:31.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-23T08:15:31.157 INFO:tasks.workunit.client.0.trial186.stderr:10 2026-02-23T08:15:31.157 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:15:31.157 INFO:tasks.workunit.client.0.trial186.stderr:1' 2026-02-23T08:15:31.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-23T08:15:31.158 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:31.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:31.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-23T08:15:31.404 INFO:tasks.workunit.client.0.trial186.stderr:osd.7 is already in. 2026-02-23T08:15:31.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:15:31.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-23T08:15:31.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-23T08:15:31.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-23T08:15:31.702 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-isa' does not exist 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-23T08:15:31.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-23T08:15:31.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-23T08:15:31.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:31.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:31.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-23T08:15:31.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-23T08:15:31.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-23T08:15:31.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:31.759 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-23T08:15:31.759 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:31.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr:9' 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-23T08:15:31.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-23T08:15:32.223 INFO:tasks.workunit.client.0.trial186.stderr:osd.9 is already out. 2026-02-23T08:15:32.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-23T08:15:37.236 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-23T08:15:37.236 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-23T08:15:37.236 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:37.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-23T08:15:37.237 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-23T08:15:37.237 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:37.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-23T08:15:37.455 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:15:37.455 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:37.455 INFO:tasks.workunit.client.0.trial186.stderr:5' 2026-02-23T08:15:37.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-23T08:15:37.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:37.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:37.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-23T08:15:37.743 INFO:tasks.workunit.client.0.trial186.stderr:osd.9 is already in. 2026-02-23T08:15:37.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:15:37.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-23T08:15:37.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-23T08:15:37.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-23T08:15:38.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-23T08:15:38.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-23T08:15:38.411 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:15:38.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:15:39.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-23T08:15:39.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:39.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-23T08:15:39.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:15:39.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-23T08:15:39.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-23T08:15:39.445 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:39.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:39.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-23T08:15:39.473 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-23T08:15:39.473 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:15:39.473 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:39.473 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-23T08:15:39.474 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:39.686 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-23T08:15:39.686 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-23T08:15:39.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-23T08:15:39.935 INFO:tasks.workunit.client.0.trial186.stderr:osd.10 is already out. 2026-02-23T08:15:39.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-23T08:15:44.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-23T08:15:44.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:15:44.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:44.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-23T08:15:44.948 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-23T08:15:44.948 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-23T08:15:45.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:45.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:45.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-23T08:15:45.417 INFO:tasks.workunit.client.0.trial186.stderr:osd.10 is already in. 2026-02-23T08:15:45.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:15:45.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-23T08:15:45.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-23T08:15:45.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-23T08:15:45.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-23T08:15:45.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-23T08:15:45.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-23T08:15:45.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:45.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:45.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-23T08:15:45.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-23T08:15:45.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:15:45.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:15:45.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:15:45.674 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:15:45.674 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:15:45.674 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:15:45.674 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:15:45.674 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:15:45.674 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:15:45.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:15:45.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:15:45.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:15:45.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:15:45.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:15:45.780 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:15:45.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:15:45.989 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:45.990 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:45.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:15:45.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:45.990 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:15:46.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836516 2026-02-23T08:15:46.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836516 2026-02-23T08:15:46.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516' 2026-02-23T08:15:46.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.068 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:15:46.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705699 2026-02-23T08:15:46.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705699 2026-02-23T08:15:46.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699' 2026-02-23T08:15:46.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:15:46.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574882 2026-02-23T08:15:46.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574882 2026-02-23T08:15:46.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882' 2026-02-23T08:15:46.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.228 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:15:46.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808854 2026-02-23T08:15:46.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808854 2026-02-23T08:15:46.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854' 2026-02-23T08:15:46.310 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.310 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:15:46.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280545 2026-02-23T08:15:46.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280545 2026-02-23T08:15:46.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854 4-94489280545' 2026-02-23T08:15:46.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.388 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:15:46.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645330 2026-02-23T08:15:46.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645330 2026-02-23T08:15:46.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854 4-94489280545 5-309237645330' 2026-02-23T08:15:46.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:15:46.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220174 2026-02-23T08:15:46.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220174 2026-02-23T08:15:46.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854 4-94489280545 5-309237645330 6-365072220174' 2026-02-23T08:15:46.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:15:46.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822687 2026-02-23T08:15:46.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822687 2026-02-23T08:15:46.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854 4-94489280545 5-309237645330 6-365072220174 7-154618822687' 2026-02-23T08:15:46.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:15:46.705 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659167 2026-02-23T08:15:46.705 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659167 2026-02-23T08:15:46.706 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854 4-94489280545 5-309237645330 6-365072220174 7-154618822687 8-176093659167' 2026-02-23T08:15:46.706 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.706 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:15:46.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056651 2026-02-23T08:15:46.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056651 2026-02-23T08:15:46.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854 4-94489280545 5-309237645330 6-365072220174 7-154618822687 8-176093659167 9-386547056651' 2026-02-23T08:15:46.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:46.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:15:46.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364829 2026-02-23T08:15:46.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364829 2026-02-23T08:15:46.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836516 1-38654705699 2-55834574882 3-287762808854 4-94489280545 5-309237645330 6-365072220174 7-154618822687 8-176093659167 9-386547056651 10-214748364829' 2026-02-23T08:15:46.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:46.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836516 2026-02-23T08:15:46.866 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:46.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:15:46.867 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836516 2026-02-23T08:15:46.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:46.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836516 2026-02-23T08:15:46.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836516' 2026-02-23T08:15:46.869 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 21474836516 2026-02-23T08:15:46.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:47.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836516 -lt 21474836516 2026-02-23T08:15:47.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:47.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705699 2026-02-23T08:15:47.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:47.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:15:47.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705699 2026-02-23T08:15:47.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:47.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705699 2026-02-23T08:15:47.090 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 38654705699 2026-02-23T08:15:47.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705699' 2026-02-23T08:15:47.091 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:15:47.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705699 -lt 38654705699 2026-02-23T08:15:47.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:47.312 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574882 2026-02-23T08:15:47.312 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:47.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:15:47.314 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574882 2026-02-23T08:15:47.314 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:47.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574882 2026-02-23T08:15:47.315 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 55834574882 2026-02-23T08:15:47.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574882' 2026-02-23T08:15:47.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:15:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574882 -lt 55834574882 2026-02-23T08:15:47.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:47.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-287762808854 2026-02-23T08:15:47.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:47.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:15:47.531 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808854 2026-02-23T08:15:47.531 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:47.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808854 2026-02-23T08:15:47.532 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 287762808854 2026-02-23T08:15:47.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 287762808854' 2026-02-23T08:15:47.533 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:15:47.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808854 -lt 287762808854 2026-02-23T08:15:47.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:47.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280545 2026-02-23T08:15:47.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:47.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:15:47.748 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280545 2026-02-23T08:15:47.748 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:47.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280545 2026-02-23T08:15:47.750 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 94489280545 2026-02-23T08:15:47.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280545' 2026-02-23T08:15:47.750 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:15:47.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280545 -lt 94489280545 2026-02-23T08:15:47.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:47.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-309237645330 2026-02-23T08:15:47.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:47.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:15:47.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645330 2026-02-23T08:15:47.985 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:47.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=309237645330 2026-02-23T08:15:47.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 309237645330' 2026-02-23T08:15:47.986 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 309237645330 2026-02-23T08:15:47.987 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:15:48.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 309237645330 -lt 309237645330 2026-02-23T08:15:48.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:48.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-365072220174 2026-02-23T08:15:48.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:48.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:15:48.205 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-365072220174 2026-02-23T08:15:48.205 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:48.206 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220174 2026-02-23T08:15:48.207 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 365072220174 2026-02-23T08:15:48.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 365072220174' 2026-02-23T08:15:48.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:15:48.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220174 -lt 365072220174 2026-02-23T08:15:48.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:48.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822687 2026-02-23T08:15:48.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:48.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:15:48.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822687 2026-02-23T08:15:48.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:48.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822687 2026-02-23T08:15:48.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822687' 2026-02-23T08:15:48.421 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.7 seq 154618822687 2026-02-23T08:15:48.421 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:15:48.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822687 -lt 154618822687 2026-02-23T08:15:48.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:48.643 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659167 2026-02-23T08:15:48.643 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:48.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:15:48.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659167 2026-02-23T08:15:48.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:48.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659167 2026-02-23T08:15:48.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659167' 2026-02-23T08:15:48.647 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.8 seq 176093659167 2026-02-23T08:15:48.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:15:48.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659167 -lt 176093659167 2026-02-23T08:15:48.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:48.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-386547056651 2026-02-23T08:15:48.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:48.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:15:48.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-386547056651 2026-02-23T08:15:48.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:48.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=386547056651 2026-02-23T08:15:48.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 386547056651' 2026-02-23T08:15:48.867 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.9 seq 386547056651 2026-02-23T08:15:48.867 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:15:49.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 386547056651 -lt 386547056651 2026-02-23T08:15:49.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:49.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364829 2026-02-23T08:15:49.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:49.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:15:49.091 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364829 2026-02-23T08:15:49.091 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:49.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364829 2026-02-23T08:15:49.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364829' 2026-02-23T08:15:49.092 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.10 seq 214748364829 2026-02-23T08:15:49.092 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:15:49.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364829 -lt 214748364829 2026-02-23T08:15:49.320 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:15:49.320 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:49.320 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:49.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-23T08:15:49.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:15:49.603 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:15:49.603 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:15:49.603 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:15:49.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:15:49.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:15:49.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:15:49.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-23T08:15:49.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:15:49.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:49.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:50.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-23T08:15:50.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:15:50.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:15:50.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-23T08:15:50.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:15:50.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:50.115 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-23T08:15:50.115 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:50.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-23T08:15:50.334 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:50.334 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-23T08:15:50.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-23T08:15:50.581 INFO:tasks.workunit.client.0.trial186.stderr:osd.5 is already out. 2026-02-23T08:15:50.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-23T08:15:50.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-23T08:15:50.890 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 is already out. 2026-02-23T08:15:50.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-23T08:15:50.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:15:50.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:15:50.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:15:50.901 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:15:50.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:15:50.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:15:50.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:15:50.903 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:15:50.903 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:15:51.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:15:51.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:15:51.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:15:51.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:15:51.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:15:51.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:15:51.213 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:51.214 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:51.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:15:51.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:15:51.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836519 2026-02-23T08:15:51.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836519 2026-02-23T08:15:51.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519' 2026-02-23T08:15:51.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.296 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:15:51.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705702 2026-02-23T08:15:51.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705702 2026-02-23T08:15:51.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702' 2026-02-23T08:15:51.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.376 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:15:51.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574885 2026-02-23T08:15:51.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574885 2026-02-23T08:15:51.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885' 2026-02-23T08:15:51.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.456 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:15:51.534 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808857 2026-02-23T08:15:51.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808857 2026-02-23T08:15:51.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857' 2026-02-23T08:15:51.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.535 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:15:51.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280548 2026-02-23T08:15:51.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280548 2026-02-23T08:15:51.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857 4-94489280548' 2026-02-23T08:15:51.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.616 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:15:51.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645333 2026-02-23T08:15:51.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645333 2026-02-23T08:15:51.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857 4-94489280548 5-309237645333' 2026-02-23T08:15:51.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.698 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:15:51.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220177 2026-02-23T08:15:51.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220177 2026-02-23T08:15:51.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857 4-94489280548 5-309237645333 6-365072220177' 2026-02-23T08:15:51.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:15:51.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822690 2026-02-23T08:15:51.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822690 2026-02-23T08:15:51.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857 4-94489280548 5-309237645333 6-365072220177 7-154618822690' 2026-02-23T08:15:51.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.856 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:15:51.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659170 2026-02-23T08:15:51.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659170 2026-02-23T08:15:51.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857 4-94489280548 5-309237645333 6-365072220177 7-154618822690 8-176093659170' 2026-02-23T08:15:51.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:51.935 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:15:52.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056654 2026-02-23T08:15:52.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056654 2026-02-23T08:15:52.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857 4-94489280548 5-309237645333 6-365072220177 7-154618822690 8-176093659170 9-386547056654' 2026-02-23T08:15:52.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:52.014 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:15:52.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364832 2026-02-23T08:15:52.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364832 2026-02-23T08:15:52.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836519 1-38654705702 2-55834574885 3-287762808857 4-94489280548 5-309237645333 6-365072220177 7-154618822690 8-176093659170 9-386547056654 10-214748364832' 2026-02-23T08:15:52.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:52.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836519 2026-02-23T08:15:52.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:52.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:15:52.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836519 2026-02-23T08:15:52.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:52.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836519 2026-02-23T08:15:52.099 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 21474836519 2026-02-23T08:15:52.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836519' 2026-02-23T08:15:52.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:52.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836517 -lt 21474836519 2026-02-23T08:15:52.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:15:53.316 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:15:53.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:53.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836519 -lt 21474836519 2026-02-23T08:15:53.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:53.537 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705702 2026-02-23T08:15:53.537 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:53.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:15:53.539 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705702 2026-02-23T08:15:53.539 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:53.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705702 2026-02-23T08:15:53.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705702' 2026-02-23T08:15:53.541 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 38654705702 2026-02-23T08:15:53.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:15:53.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705702 -lt 38654705702 2026-02-23T08:15:53.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:53.773 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574885 2026-02-23T08:15:53.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:53.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:15:53.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574885 2026-02-23T08:15:53.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:53.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574885 2026-02-23T08:15:53.777 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 55834574885 2026-02-23T08:15:53.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574885' 2026-02-23T08:15:53.777 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:15:54.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574885 -lt 55834574885 2026-02-23T08:15:54.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:54.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-287762808857 2026-02-23T08:15:54.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:54.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:15:54.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808857 2026-02-23T08:15:54.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:54.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808857 2026-02-23T08:15:54.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 287762808857' 2026-02-23T08:15:54.007 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 287762808857 2026-02-23T08:15:54.007 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:15:54.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808857 -lt 287762808857 2026-02-23T08:15:54.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:54.227 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280548 2026-02-23T08:15:54.227 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:54.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:15:54.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280548 2026-02-23T08:15:54.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:54.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280548 2026-02-23T08:15:54.231 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 94489280548 2026-02-23T08:15:54.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280548' 2026-02-23T08:15:54.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:15:54.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280548 -lt 94489280548 2026-02-23T08:15:54.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:54.448 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-309237645333 2026-02-23T08:15:54.448 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:54.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:15:54.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645333 2026-02-23T08:15:54.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:54.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=309237645333 2026-02-23T08:15:54.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 309237645333' 2026-02-23T08:15:54.451 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 309237645333 2026-02-23T08:15:54.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:15:54.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 309237645334 -lt 309237645333 2026-02-23T08:15:54.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:54.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-365072220177 2026-02-23T08:15:54.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:54.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:15:54.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-365072220177 2026-02-23T08:15:54.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:54.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220177 2026-02-23T08:15:54.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 365072220177' 2026-02-23T08:15:54.671 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 365072220177 2026-02-23T08:15:54.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:15:54.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220178 -lt 365072220177 2026-02-23T08:15:54.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:54.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822690 2026-02-23T08:15:54.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:54.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:15:54.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822690 2026-02-23T08:15:54.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:54.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822690 2026-02-23T08:15:54.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822690' 2026-02-23T08:15:54.896 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.7 seq 154618822690 2026-02-23T08:15:54.896 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:15:55.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822691 -lt 154618822690 2026-02-23T08:15:55.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:55.123 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659170 2026-02-23T08:15:55.123 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:55.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:15:55.124 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659170 2026-02-23T08:15:55.125 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:55.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659170 2026-02-23T08:15:55.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659170' 2026-02-23T08:15:55.126 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.8 seq 176093659170 2026-02-23T08:15:55.127 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:15:55.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659170 -lt 176093659170 2026-02-23T08:15:55.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:55.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-386547056654 2026-02-23T08:15:55.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:55.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:15:55.340 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-386547056654 2026-02-23T08:15:55.340 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:55.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=386547056654 2026-02-23T08:15:55.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 386547056654' 2026-02-23T08:15:55.342 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.9 seq 386547056654 2026-02-23T08:15:55.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:15:55.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 386547056654 -lt 386547056654 2026-02-23T08:15:55.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:55.573 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364832 2026-02-23T08:15:55.573 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:55.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:15:55.575 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364832 2026-02-23T08:15:55.575 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:55.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364832 2026-02-23T08:15:55.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364832' 2026-02-23T08:15:55.577 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.10 seq 214748364832 2026-02-23T08:15:55.577 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:15:55.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364833 -lt 214748364832 2026-02-23T08:15:55.798 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:15:55.799 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:55.799 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:15:56.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:15:56.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-23T08:15:56.315 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:15:56.315 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:15:56.315 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:15:56.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-23T08:15:56.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:15:56.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:15:56.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-23T08:15:56.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-23T08:15:56.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:15:56.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:56.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-23T08:15:56.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-23T08:15:56.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:56.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-23T08:15:56.839 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:56.839 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:56.839 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:56.839 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:56.839 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:56.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-23T08:15:56.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-23T08:15:56.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-23T08:15:56.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:15:56.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:15:56.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-23T08:15:56.841 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-23T08:15:56.841 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:15:57.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-23T08:15:57.061 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:57.061 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:57.062 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:57.062 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:57.062 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:57.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-23T08:15:57.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:15:57.079 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:15:57.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-23T08:15:57.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-23T08:15:57.352 INFO:tasks.workunit.client.0.trial186.stderr:osd.5 is already in. 2026-02-23T08:15:57.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-23T08:15:57.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-23T08:15:57.658 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 is already in. 2026-02-23T08:15:57.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-23T08:15:57.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:15:57.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:15:57.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:15:57.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:15:57.670 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:15:57.670 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:15:57.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:15:57.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:15:57.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:15:57.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:15:57.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:15:57.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:15:57.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:15:57.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:15:57.777 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:15:57.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:15:57.997 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:8 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:15:57.998 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:15:57.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:15:57.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:57.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:15:58.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836522 2026-02-23T08:15:58.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836522 2026-02-23T08:15:58.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522' 2026-02-23T08:15:58.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:15:58.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705705 2026-02-23T08:15:58.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705705 2026-02-23T08:15:58.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705' 2026-02-23T08:15:58.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.157 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:15:58.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574889 2026-02-23T08:15:58.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574889 2026-02-23T08:15:58.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889' 2026-02-23T08:15:58.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.239 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:15:58.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808860 2026-02-23T08:15:58.316 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808860 2026-02-23T08:15:58.316 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860' 2026-02-23T08:15:58.316 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:15:58.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=94489280551 2026-02-23T08:15:58.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 94489280551 2026-02-23T08:15:58.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860 4-94489280551' 2026-02-23T08:15:58.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.396 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:15:58.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645337 2026-02-23T08:15:58.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645337 2026-02-23T08:15:58.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860 4-94489280551 5-309237645337' 2026-02-23T08:15:58.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.475 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:15:58.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=365072220180 2026-02-23T08:15:58.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 365072220180 2026-02-23T08:15:58.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860 4-94489280551 5-309237645337 6-365072220180' 2026-02-23T08:15:58.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.554 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:15:58.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822694 2026-02-23T08:15:58.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822694 2026-02-23T08:15:58.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860 4-94489280551 5-309237645337 6-365072220180 7-154618822694' 2026-02-23T08:15:58.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:15:58.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659173 2026-02-23T08:15:58.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659173 2026-02-23T08:15:58.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860 4-94489280551 5-309237645337 6-365072220180 7-154618822694 8-176093659173' 2026-02-23T08:15:58.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.712 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-23T08:15:58.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056657 2026-02-23T08:15:58.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056657 2026-02-23T08:15:58.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860 4-94489280551 5-309237645337 6-365072220180 7-154618822694 8-176093659173 9-386547056657' 2026-02-23T08:15:58.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:15:58.792 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-23T08:15:58.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364835 2026-02-23T08:15:58.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364835 2026-02-23T08:15:58.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836522 1-38654705705 2-55834574889 3-287762808860 4-94489280551 5-309237645337 6-365072220180 7-154618822694 8-176093659173 9-386547056657 10-214748364835' 2026-02-23T08:15:58.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:58.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836522 2026-02-23T08:15:58.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:58.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:15:58.874 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836522 2026-02-23T08:15:58.874 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:58.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836522 2026-02-23T08:15:58.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836522' 2026-02-23T08:15:58.876 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 21474836522 2026-02-23T08:15:58.876 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:15:59.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836522 -lt 21474836522 2026-02-23T08:15:59.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:59.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705705 2026-02-23T08:15:59.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:59.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:15:59.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705705 2026-02-23T08:15:59.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:59.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705705 2026-02-23T08:15:59.099 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 38654705705 2026-02-23T08:15:59.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705705' 2026-02-23T08:15:59.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:15:59.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705705 -lt 38654705705 2026-02-23T08:15:59.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:59.314 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574889 2026-02-23T08:15:59.314 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:59.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:15:59.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574889 2026-02-23T08:15:59.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:59.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574889 2026-02-23T08:15:59.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574889' 2026-02-23T08:15:59.318 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 55834574889 2026-02-23T08:15:59.318 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:15:59.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574889 -lt 55834574889 2026-02-23T08:15:59.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:59.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-287762808860 2026-02-23T08:15:59.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:59.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:15:59.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808860 2026-02-23T08:15:59.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:59.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808860 2026-02-23T08:15:59.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 287762808860' 2026-02-23T08:15:59.552 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 287762808860 2026-02-23T08:15:59.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:15:59.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808860 -lt 287762808860 2026-02-23T08:15:59.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:15:59.778 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-94489280551 2026-02-23T08:15:59.779 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:15:59.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:15:59.780 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-94489280551 2026-02-23T08:15:59.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:15:59.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=94489280551 2026-02-23T08:15:59.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 94489280551' 2026-02-23T08:15:59.782 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 94489280551 2026-02-23T08:15:59.783 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:15:59.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 94489280551 -lt 94489280551 2026-02-23T08:15:59.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:00.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-309237645337 2026-02-23T08:16:00.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:00.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:16:00.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645337 2026-02-23T08:16:00.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:00.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=309237645337 2026-02-23T08:16:00.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 309237645337' 2026-02-23T08:16:00.004 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 309237645337 2026-02-23T08:16:00.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:16:00.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 309237645337 -lt 309237645337 2026-02-23T08:16:00.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:00.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-365072220180 2026-02-23T08:16:00.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:00.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:16:00.224 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-365072220180 2026-02-23T08:16:00.224 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:00.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=365072220180 2026-02-23T08:16:00.226 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 365072220180 2026-02-23T08:16:00.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 365072220180' 2026-02-23T08:16:00.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:16:00.452 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 365072220180 -lt 365072220180 2026-02-23T08:16:00.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:00.453 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-154618822694 2026-02-23T08:16:00.453 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:00.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:16:00.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-154618822694 2026-02-23T08:16:00.456 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:00.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822694 2026-02-23T08:16:00.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 154618822694' 2026-02-23T08:16:00.457 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.7 seq 154618822694 2026-02-23T08:16:00.458 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:16:00.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822694 -lt 154618822694 2026-02-23T08:16:00.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:00.681 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-176093659173 2026-02-23T08:16:00.681 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:00.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:16:00.683 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-176093659173 2026-02-23T08:16:00.683 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:00.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659173 2026-02-23T08:16:00.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 176093659173' 2026-02-23T08:16:00.685 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.8 seq 176093659173 2026-02-23T08:16:00.686 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:16:00.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659173 -lt 176093659173 2026-02-23T08:16:00.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:00.909 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-386547056657 2026-02-23T08:16:00.909 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:00.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-23T08:16:00.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-386547056657 2026-02-23T08:16:00.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:00.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=386547056657 2026-02-23T08:16:00.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 386547056657' 2026-02-23T08:16:00.913 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.9 seq 386547056657 2026-02-23T08:16:00.913 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-23T08:16:01.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 386547056657 -lt 386547056657 2026-02-23T08:16:01.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:01.139 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-214748364835 2026-02-23T08:16:01.140 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:01.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-23T08:16:01.141 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364835 2026-02-23T08:16:01.142 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:01.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364835 2026-02-23T08:16:01.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 214748364835' 2026-02-23T08:16:01.144 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.10 seq 214748364835 2026-02-23T08:16:01.144 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-23T08:16:01.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364836 -lt 214748364835 2026-02-23T08:16:01.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:16:01.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:16:01.361 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:16:01.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-23T08:16:01.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:16:01.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:16:01.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:16:01.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:16:01.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:16:01.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:16:01.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:16:01.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-23T08:16:01.871 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:16:01.871 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:16:01.871 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:16:02.158 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-23T08:16:02.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:16:02.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:16:02.159 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-23T08:16:02.159 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:16:02.159 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:16:02.160 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-23T08:16:02.160 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:02.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-23T08:16:02.373 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:16:02.373 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:16:02.374 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:16:02.374 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:16:02.374 INFO:tasks.workunit.client.0.trial186.stderr:10' 2026-02-23T08:16:02.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-23T08:16:02.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-23T08:16:02.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:16:02.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-23T08:16:02.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-23T08:16:02.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:16:02.618 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:16:02.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-23T08:16:02.910 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-23T08:16:02.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-23T08:16:02.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-23T08:16:02.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-23T08:16:02.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-23T08:16:02.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-23T08:16:02.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-23T08:16:03.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-23T08:16:03.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-23T08:16:03.609 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-lrc-a' already exists 2026-02-23T08:16:03.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:16:04.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-23T08:16:04.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-23T08:16:04.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-23T08:16:04.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-23T08:16:04.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:16:04.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:16:04.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-23T08:16:04.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-23T08:16:04.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-23T08:16:04.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:16:04.673 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-23T08:16:04.673 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:04.898 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-23T08:16:04.898 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:16:04.898 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:16:04.899 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-23T08:16:04.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-23T08:16:04.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-23T08:16:04.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-23T08:16:04.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-23T08:16:05.144 INFO:tasks.workunit.client.0.trial186.stderr:osd.3 is already out. 2026-02-23T08:16:05.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-23T08:16:10.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-23T08:16:10.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-23T08:16:10.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:16:10.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-23T08:16:10.157 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-23T08:16:10.158 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:10.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-23T08:16:10.378 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:16:10.378 INFO:tasks.workunit.client.0.trial186.stderr:7' 2026-02-23T08:16:10.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-23T08:16:10.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:16:10.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:16:10.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-23T08:16:10.627 INFO:tasks.workunit.client.0.trial186.stderr:osd.3 is already in. 2026-02-23T08:16:10.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:16:10.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-23T08:16:10.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-23T08:16:10.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-23T08:16:10.929 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-lrc-a' does not exist 2026-02-23T08:16:10.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-23T08:16:11.214 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-23T08:16:11.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-23T08:16:11.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-23T08:16:11.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-23T08:16:11.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-23T08:16:11.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-23T08:16:11.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-23T08:16:11.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-23T08:16:11.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-23T08:16:11.910 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-lrc' already exists 2026-02-23T08:16:11.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:16:12.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-23T08:16:12.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-23T08:16:12.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-23T08:16:12.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-23T08:16:12.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:16:12.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:16:12.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-23T08:16:12.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-23T08:16:12.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-23T08:16:12.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:16:12.970 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-23T08:16:12.970 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:13.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-23T08:16:13.187 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:16:13.187 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:10 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:7' 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-23T08:16:13.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-23T08:16:13.436 INFO:tasks.workunit.client.0.trial186.stderr:osd.7 is already out. 2026-02-23T08:16:13.446 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-23T08:16:18.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-23T08:16:18.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-23T08:16:18.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:16:18.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-23T08:16:18.449 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-23T08:16:18.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:18.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-23T08:16:18.662 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:16:18.662 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:16:18.662 INFO:tasks.workunit.client.0.trial186.stderr:10 2026-02-23T08:16:18.663 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:16:18.663 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:16:18.663 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:16:18.663 INFO:tasks.workunit.client.0.trial186.stderr:4' 2026-02-23T08:16:18.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-23T08:16:18.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:16:18.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:16:18.681 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-23T08:16:18.915 INFO:tasks.workunit.client.0.trial186.stderr:osd.7 is already in. 2026-02-23T08:16:18.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:16:18.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-23T08:16:18.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-23T08:16:18.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-23T08:16:19.212 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-lrc' does not exist 2026-02-23T08:16:19.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-23T08:16:19.519 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile profile-lrc does not exist 2026-02-23T08:16:19.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-23T08:16:19.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-23T08:16:19.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-23T08:16:19.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-23T08:16:19.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-23T08:16:19.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-23T08:16:19.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-23T08:16:19.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-23T08:16:20.202 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-shec' already exists 2026-02-23T08:16:20.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:16:21.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-23T08:16:21.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-23T08:16:21.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-23T08:16:21.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-23T08:16:21.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-23T08:16:21.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-23T08:16:21.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:16:21.261 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:16:21.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-23T08:16:21.264 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-23T08:16:21.264 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-23T08:16:21.264 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:16:21.265 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-23T08:16:21.265 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:21.475 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-23T08:16:21.476 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:16:21.476 INFO:tasks.workunit.client.0.trial186.stderr:1' 2026-02-23T08:16:21.476 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-23T08:16:21.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-23T08:16:21.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-23T08:16:21.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-23T08:16:21.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-23T08:16:21.744 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 is already out. 2026-02-23T08:16:21.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-23T08:16:26.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-23T08:16:26.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-23T08:16:26.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-23T08:16:26.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-23T08:16:26.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-23T08:16:26.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:26.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-23T08:16:26.980 INFO:tasks.workunit.client.0.trial186.stderr:9 2026-02-23T08:16:26.980 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:16:26.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-23T08:16:26.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-23T08:16:26.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-23T08:16:26.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-23T08:16:27.223 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 is already in. 2026-02-23T08:16:27.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-23T08:16:27.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-23T08:16:27.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-23T08:16:27.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-23T08:16:27.528 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-shec' does not exist 2026-02-23T08:16:27.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-23T08:16:27.807 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile profile-shec does not exist 2026-02-23T08:16:27.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-23T08:16:27.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-23T08:16:27.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-23T08:16:28.081 INFO:tasks.workunit.client.0.trial186.stderr:pool 'ecpool' does not exist 2026-02-23T08:16:28.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-23T08:16:28.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-23T08:16:28.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:16:28.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-23T08:16:28.093 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:16:28.093 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:16:28.093 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:16:28.093 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:16:28.093 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:16:28.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:16:28.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:16:28.408 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:16:28.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:16:28.411 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:16:28.411 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:16:28.411 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:16:28.412 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:16:28.412 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:16:28.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:16:28.413 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:16:28.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:16:28.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:16:28.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-23T08:16:28.474 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:16:28.474 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.474 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:16:28.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46012 2026-02-23T08:16:28.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:16:28.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:16:28.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-23T08:16:28.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code 0 2026-02-23T08:16:28.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-23T08:16:28.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-23T08:16:28.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-23T08:16:28.476 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:16:28.476 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:16:28.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:16:28.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:16:28.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:16:28.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:16:28.479 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:16:28.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:16:28.480 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:16:28.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:16:28.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:16:28.481 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:16:28.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:16:28.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:16:28.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:16:28.483 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:16:28.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:16:28.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-23T08:16:28.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-23T08:16:28.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:16:28.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.46012 2026-02-23T08:16:28.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.46012 2026-02-23T08:16:28.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:16:28.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:16:28.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-23T08:16:28.488 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-23T08:16:28.488 DEBUG:teuthology.orchestra.run.trial186:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-23T08:16:28.543 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-23T08:16:28.543 DEBUG:teuthology.orchestra.run.trial186:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5adce3ff0fe1645dcbc38d059f948571dc3d69da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-23T08:16:28.592 INFO:tasks.workunit.client.0.trial186.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-eio 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-23T08:16:28.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-23T08:16:28.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none ' 2026-02-23T08:16:28.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-23T08:16:28.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-23T08:16:28.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-23T08:16:28.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_ec_object_attr_read_error 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_ec_recovery_multiple_errors 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_ec_recovery_multiple_objects 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_ec_recovery_unfound 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_ec_single_recovery_error 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_get_bad_size_shard_0 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_get_bad_size_shard_1 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_get_subread_missing 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:16:28.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:16:28.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:16:28.603 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:16:28.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:16:28.605 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:16:28.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:16:28.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:16:28.606 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:16:28.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:16:28.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:16:28.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:16:28.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:16:28.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:16:28.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:16:28.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:16:28.611 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:16:28.611 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.611 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:28.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:16:28.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:16:28.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:16:28.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:16:28.613 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:16:28.613 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.613 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:28.613 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:16:28.615 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:16:28.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:16:28.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:16:28.637 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:16:28.637 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:16:28.638 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:16:28.638 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:16:28.638 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.638 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:28.638 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:28.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:16:28.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:16:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:16:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:16:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:16:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:16:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:16:28.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:16:28.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:16:28.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:16:28.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:16:28.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:16:28.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:16:28.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:16:28.723 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:16:28.724 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:16:28.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:16:28.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:16:28.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:16:28.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:16:28.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:16:28.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:16:28.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:16:28.886 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:16:28.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:16:28.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:16:28.887 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:16:28.887 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:28.887 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:28.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:28.888 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:16:28.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:16:28.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:16:28.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:16:29.016 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:16:29.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:16:30.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:16:30.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:16:30.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:16:30.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:16:30.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:30.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:30.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:16:30.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:16:30.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:16:30.075 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:16:30.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:16:30.083 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:16:28.655+0000 7f7d9decdec0 0 load: jerasure load: lrc 2026-02-23T08:16:30.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:16:30.085 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-23T08:16:30.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-23T08:16:30.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:16:30.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:16:30.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:16:30.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:16:30.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:16:30.090 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:16:30.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:16:30.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:16:30.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:16:30.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:16:30.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:16:30.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:16:30.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:16:30.092 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:16:30.093 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d36423b1-d7a0-43c8-bed0-b6d0533ff0a0 2026-02-23T08:16:30.093 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d36423b1-d7a0-43c8-bed0-b6d0533ff0a0' 2026-02-23T08:16:30.093 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 d36423b1-d7a0-43c8-bed0-b6d0533ff0a0 2026-02-23T08:16:30.093 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:16:30.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDeDJxpsTUSBhAAaIbUKftF6fbsfT2ELgcIYw== 2026-02-23T08:16:30.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDeDJxpsTUSBhAAaIbUKftF6fbsfT2ELgcIYw=="}' 2026-02-23T08:16:30.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d36423b1-d7a0-43c8-bed0-b6d0533ff0a0 -i td/test-erasure-eio/0/new.json 2026-02-23T08:16:30.214 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:16:30.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:16:30.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDeDJxpsTUSBhAAaIbUKftF6fbsfT2ELgcIYw== --osd-uuid d36423b1-d7a0-43c8-bed0-b6d0533ff0a0 2026-02-23T08:16:30.239 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:30.235+0000 7fbb6b153a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:30.240 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:30.239+0000 7fbb6b153a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:30.241 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:30.239+0000 7fbb6b153a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:30.241 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:30.239+0000 7fbb6b153a40 -1 bdev(0x563ad4488400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:16:30.242 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:30.239+0000 7fbb6b153a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:16:30.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:16:30.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:16:30.943 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:16:30.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:16:30.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:16:31.066 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:16:31.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:16:31.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:16:31.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:16:31.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:16:31.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:16:31.119 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:31.115+0000 7f2d74ee7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:31.120 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:31.119+0000 7f2d74ee7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:31.121 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:31.119+0000 7f2d74ee7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:31.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:16:31.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:16:31.509 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:31.507+0000 7f2d74ee7a40 -1 Falling back to public interface 2026-02-23T08:16:32.073 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:32.071+0000 7f2d74ee7a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:16:32.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:16:32.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:32.335 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:16:32.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:16:32.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:32.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:16:32.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:16:33.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:16:33.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:33.556 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:16:33.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:16:33.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:33.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:16:33.778 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/806103574,v1:127.0.0.1:6803/806103574] [v2:127.0.0.1:6804/806103574,v1:127.0.0.1:6805/806103574] exists,up d36423b1-d7a0-43c8-bed0-b6d0533ff0a0 2026-02-23T08:16:33.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:16:33.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:16:33.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:16:33.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:33.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:16:33.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:16:33.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:16:33.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=09c057ab-3ef1-41b2-b2f7-6710853c3601 2026-02-23T08:16:33.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 09c057ab-3ef1-41b2-b2f7-6710853c3601' 2026-02-23T08:16:33.786 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 09c057ab-3ef1-41b2-b2f7-6710853c3601 2026-02-23T08:16:33.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:16:33.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDhDJxpKOVoLxAA0b0WYDHEGR9ASzy0dALpSQ== 2026-02-23T08:16:33.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDhDJxpKOVoLxAA0b0WYDHEGR9ASzy0dALpSQ=="}' 2026-02-23T08:16:33.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 09c057ab-3ef1-41b2-b2f7-6710853c3601 -i td/test-erasure-eio/1/new.json 2026-02-23T08:16:34.016 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:16:34.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:16:34.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDhDJxpKOVoLxAA0b0WYDHEGR9ASzy0dALpSQ== --osd-uuid 09c057ab-3ef1-41b2-b2f7-6710853c3601 2026-02-23T08:16:34.043 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:34.039+0000 7f2a00ff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:34.045 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:34.043+0000 7f2a00ff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:34.046 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:34.043+0000 7f2a00ff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:34.046 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:34.043+0000 7f2a00ff7a40 -1 bdev(0x55d46733e400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:16:34.047 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:34.043+0000 7f2a00ff7a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:16:34.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:16:34.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:16:34.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:16:34.687 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:16:34.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:16:34.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:16:34.984 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:16:34.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:16:34.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:16:34.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:16:34.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:16:35.010 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:35.007+0000 7f5643cd3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:35.012 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:35.011+0000 7f5643cd3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:35.013 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:35.011+0000 7f5643cd3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:35.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:16:35.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:35.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:16:35.361 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:35.359+0000 7f5643cd3a40 -1 Falling back to public interface 2026-02-23T08:16:35.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:16:36.004 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:36.003+0000 7f5643cd3a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:16:36.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:16:36.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:36.420 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:16:36.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:16:36.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:36.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:16:36.641 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2721278612,v1:127.0.0.1:6811/2721278612] [v2:127.0.0.1:6812/2721278612,v1:127.0.0.1:6813/2721278612] exists,up 09c057ab-3ef1-41b2-b2f7-6710853c3601 2026-02-23T08:16:36.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:16:36.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:16:36.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:16:36.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:16:36.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:16:36.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:16:36.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:16:36.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:16:36.648 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:16:36.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e82a04ae-5fb3-46a1-b1c6-6707eda6863c 2026-02-23T08:16:36.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 e82a04ae-5fb3-46a1-b1c6-6707eda6863c' 2026-02-23T08:16:36.649 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 e82a04ae-5fb3-46a1-b1c6-6707eda6863c 2026-02-23T08:16:36.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:16:36.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDkDJxprlQ8JxAArrPc/DyVGAO8v5/LMAO1lg== 2026-02-23T08:16:36.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDkDJxprlQ8JxAArrPc/DyVGAO8v5/LMAO1lg=="}' 2026-02-23T08:16:36.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e82a04ae-5fb3-46a1-b1c6-6707eda6863c -i td/test-erasure-eio/2/new.json 2026-02-23T08:16:36.885 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:16:36.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:16:36.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDkDJxprlQ8JxAArrPc/DyVGAO8v5/LMAO1lg== --osd-uuid e82a04ae-5fb3-46a1-b1c6-6707eda6863c 2026-02-23T08:16:36.914 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:36.911+0000 7f8ccfe46a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:36.916 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:36.915+0000 7f8ccfe46a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:36.917 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:36.915+0000 7f8ccfe46a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:36.917 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:36.915+0000 7f8ccfe46a40 -1 bdev(0x55619b3e6400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:16:36.918 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:36.915+0000 7f8ccfe46a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:16:37.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:16:37.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:16:37.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:16:37.556 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:16:37.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:16:37.849 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:16:37.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:16:37.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:16:37.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:16:37.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:16:37.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:16:37.876 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:37.875+0000 7fd613725a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:37.877 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:37.875+0000 7fd613725a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:37.878 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:37.875+0000 7fd613725a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:38.070 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:16:38.241 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:38.239+0000 7fd613725a40 -1 Falling back to public interface 2026-02-23T08:16:38.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:16:38.867 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:38.863+0000 7fd613725a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:16:39.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:16:39.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:39.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:16:39.291 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:16:39.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:39.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:16:39.517 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/871290374,v1:127.0.0.1:6819/871290374] [v2:127.0.0.1:6820/871290374,v1:127.0.0.1:6821/871290374] exists,up e82a04ae-5fb3-46a1-b1c6-6707eda6863c 2026-02-23T08:16:39.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:16:39.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:16:39.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:16:39.522 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:16:39.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:16:39.524 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:16:39.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=193e83a9-55b0-4bfb-aa50-54c9f6064edd 2026-02-23T08:16:39.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 193e83a9-55b0-4bfb-aa50-54c9f6064edd' 2026-02-23T08:16:39.526 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 193e83a9-55b0-4bfb-aa50-54c9f6064edd 2026-02-23T08:16:39.526 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:16:39.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDnDJxpk/jbHxAAYSD1Go8QHMiMINBDQ0gmMA== 2026-02-23T08:16:39.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDnDJxpk/jbHxAAYSD1Go8QHMiMINBDQ0gmMA=="}' 2026-02-23T08:16:39.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 193e83a9-55b0-4bfb-aa50-54c9f6064edd -i td/test-erasure-eio/3/new.json 2026-02-23T08:16:39.764 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:16:39.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:16:39.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDnDJxpk/jbHxAAYSD1Go8QHMiMINBDQ0gmMA== --osd-uuid 193e83a9-55b0-4bfb-aa50-54c9f6064edd 2026-02-23T08:16:39.791 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:39.787+0000 7f330e68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:39.793 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:39.791+0000 7f330e68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:39.794 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:39.791+0000 7f330e68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:39.794 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:39.791+0000 7f330e68ea40 -1 bdev(0x563ca544c400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:16:39.795 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:39.791+0000 7f330e68ea40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:16:40.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:16:40.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:16:40.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:16:40.432 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:16:40.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:16:40.729 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:16:40.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:16:40.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:16:40.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:16:40.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:16:40.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:16:40.754 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:40.751+0000 7fe8556c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:40.756 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:40.755+0000 7fe8556c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:40.757 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:40.755+0000 7fe8556c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:40.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:16:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:16:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:16:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:16:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:16:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:16:40.947 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:16:40.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:40.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:16:41.164 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:16:41.193 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:41.191+0000 7fe8556c0a40 -1 Falling back to public interface 2026-02-23T08:16:41.766 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:41.763+0000 7fe8556c0a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:16:42.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:16:42.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:42.166 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:16:42.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:16:42.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:42.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:16:42.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:16:43.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:16:43.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:43.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:16:43.380 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:16:43.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:43.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:16:43.591 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2749890807,v1:127.0.0.1:6827/2749890807] [v2:127.0.0.1:6828/2749890807,v1:127.0.0.1:6829/2749890807] exists,up 193e83a9-55b0-4bfb-aa50-54c9f6064edd 2026-02-23T08:16:43.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:43.593 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:16:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-23T08:16:43.595 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:16:43.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fc621d5b-400a-4410-9113-d632fd894d3c 2026-02-23T08:16:43.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 fc621d5b-400a-4410-9113-d632fd894d3c' 2026-02-23T08:16:43.596 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 fc621d5b-400a-4410-9113-d632fd894d3c 2026-02-23T08:16:43.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:16:43.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDrDJxpnNBSJBAAon7OHP703b6jnqFhJhxxHg== 2026-02-23T08:16:43.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDrDJxpnNBSJBAAon7OHP703b6jnqFhJhxxHg=="}' 2026-02-23T08:16:43.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fc621d5b-400a-4410-9113-d632fd894d3c -i td/test-erasure-eio/4/new.json 2026-02-23T08:16:43.845 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:16:43.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-23T08:16:43.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDrDJxpnNBSJBAAon7OHP703b6jnqFhJhxxHg== --osd-uuid fc621d5b-400a-4410-9113-d632fd894d3c 2026-02-23T08:16:43.874 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:43.871+0000 7fe50bd4ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:43.876 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:43.875+0000 7fe50bd4ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:43.877 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:43.875+0000 7fe50bd4ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:43.877 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:43.875+0000 7fe50bd4ba40 -1 bdev(0x55ad06680400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:16:43.877 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:43.875+0000 7fe50bd4ba40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-23T08:16:44.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-23T08:16:44.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:16:44.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:16:44.524 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:16:44.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:16:44.826 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:16:44.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:16:44.828 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:16:44.828 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:16:44.828 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:16:44.828 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:16:44.852 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:44.851+0000 7fbba875fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:44.854 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:44.851+0000 7fbba875fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:44.855 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:44.855+0000 7fbba875fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:16:45.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:16:45.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:16:45.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:16:45.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:16:45.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:16:45.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:45.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:16:45.045 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:16:45.045 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:45.045 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:16:45.225 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:45.223+0000 7fbba875fa40 -1 Falling back to public interface 2026-02-23T08:16:45.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:16:45.837 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:16:45.835+0000 7fbba875fa40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:16:46.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:16:46.272 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:16:46.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:16:46.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:16:46.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:16:46.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:16:46.500 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/412789488,v1:127.0.0.1:6835/412789488] [v2:127.0.0.1:6836/412789488,v1:127.0.0.1:6837/412789488] exists,up fc621d5b-400a-4410-9113-d632fd894d3c 2026-02-23T08:16:46.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:16:46.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:16:46.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:16:46.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:16:46.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:16:46.558 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:16:46.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:16:46.566 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:16:31.511+0000 7f2d74ee7a40 0 load: jerasure load: lrc 2026-02-23T08:16:46.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:16:46.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-23T08:16:46.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-23T08:16:46.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:16:46.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:16:46.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-23T08:16:46.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:16:46.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-23T08:16:46.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:16:46.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-23T08:16:46.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:16:46.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:16:47.215 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:16:47.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:16:48.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:16:48.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:16:48.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:16:48.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:16:48.227 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:16:48.228 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:16:48.228 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:16:48.228 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:16:48.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:16:48.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:16:48.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:16:48.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:16:48.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:16:48.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:16:48.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:16:48.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:16:48.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:16:48.550 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:16:48.550 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:16:48.550 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:16:48.550 INFO:tasks.workunit.client.0.trial186.stderr:4' 2026-02-23T08:16:48.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:16:48.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:16:48.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:16:48.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-23T08:16:48.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-23T08:16:48.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-23T08:16:48.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:16:48.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:16:48.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-23T08:16:48.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-23T08:16:48.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853' 2026-02-23T08:16:48.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:16:48.710 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:16:48.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-23T08:16:48.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-23T08:16:48.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-81604378629' 2026-02-23T08:16:48.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:16:48.790 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:16:48.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116997 2026-02-23T08:16:48.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116997 2026-02-23T08:16:48.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-81604378629 3-115964116997' 2026-02-23T08:16:48.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:16:48.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:16:48.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855363 2026-02-23T08:16:48.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855363 2026-02-23T08:16:48.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-81604378629 3-115964116997 4-150323855363' 2026-02-23T08:16:48.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:48.949 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-23T08:16:48.949 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:48.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:16:48.951 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-23T08:16:48.951 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:48.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-23T08:16:48.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-23T08:16:48.953 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803782 2026-02-23T08:16:48.953 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:16:49.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803782 2026-02-23T08:16:49.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:16:50.164 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:16:50.165 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:16:50.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-23T08:16:50.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:50.377 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-23T08:16:50.377 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:50.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:16:50.378 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-23T08:16:50.379 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:50.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-23T08:16:50.380 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574853 2026-02-23T08:16:50.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-23T08:16:50.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:16:50.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-23T08:16:50.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:50.591 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378629 2026-02-23T08:16:50.591 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:50.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:16:50.593 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378629 2026-02-23T08:16:50.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:50.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378629 2026-02-23T08:16:50.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378629' 2026-02-23T08:16:50.595 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 81604378629 2026-02-23T08:16:50.595 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:16:50.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378629 -lt 81604378629 2026-02-23T08:16:50.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:50.810 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116997 2026-02-23T08:16:50.810 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:50.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:16:50.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116997 2026-02-23T08:16:50.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:50.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116997 2026-02-23T08:16:50.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116997' 2026-02-23T08:16:50.814 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 115964116997 2026-02-23T08:16:50.814 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:16:51.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-23T08:16:51.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:16:51.044 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855363 2026-02-23T08:16:51.045 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:16:51.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:16:51.046 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855363 2026-02-23T08:16:51.047 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:16:51.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855363 2026-02-23T08:16:51.048 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 150323855363 2026-02-23T08:16:51.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855363' 2026-02-23T08:16:51.048 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:16:51.261 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-23T08:16:51.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:16:51.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:16:51.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:16:51.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:16:51.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:16:51.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:16:51.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:16:51.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:16:51.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:16:51.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:16:51.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:16:51.759 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:16:51.760 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:16:51.760 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:16:51.760 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:16:52.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:16:52.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:16:52.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:16:52.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-23T08:16:52.242 INFO:tasks.workunit.client.0.trial186.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-23T08:16:52.242 INFO:tasks.workunit.client.0.trial186.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:48.187846+0000 0'0 42:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-23T08:16:47.162192+0000 0'0 2026-02-23T08:16:47.162192+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:48:08.875198+0000 0 0 2026-02-23T08:16:52.242 INFO:tasks.workunit.client.0.trial186.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.947048+0000 0'0 41:20 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:54:33.056436+0000 0 0 2026-02-23T08:16:52.242 INFO:tasks.workunit.client.0.trial186.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.947188+0000 0'0 42:107 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:20:07.003176+0000 0 0 2026-02-23T08:16:52.242 INFO:tasks.workunit.client.0.trial186.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.733718+0000 0'0 41:20 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:56:45.414249+0000 0 0 2026-02-23T08:16:52.242 INFO:tasks.workunit.client.0.trial186.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:39.545663+0000 0'0 41:66 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T13:46:10.102710+0000 0 0 2026-02-23T08:16:52.243 INFO:tasks.workunit.client.0.trial186.stdout: 2026-02-23T08:16:52.243 INFO:tasks.workunit.client.0.trial186.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-23T08:16:52.243 INFO:tasks.workunit.client.0.trial186.stderr:dumped pgs 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:16:52.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:16:52.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:52.280 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-23T08:16:52.280 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-23T08:16:52.280 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-23T08:16:52.281 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:16:52.281 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-23T08:16:52.491 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-23T08:16:52.492 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-23T08:16:52.492 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:16:52.492 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:16:52.493 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:16:52.493 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:16:52.716 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:16:52.716 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:16:52.716 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:16:52.716 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:16:52.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:16:53.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:16:53.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-23T08:16:53.243 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already down. 2026-02-23T08:16:53.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-23T08:16:53.494 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already out. 2026-02-23T08:16:53.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-23T08:16:53.714 INFO:tasks.workunit.client.0.trial186.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-23T08:16:53.714 INFO:tasks.workunit.client.0.trial186.stdout:2.0 1 0 1 0 0 4096 0 0 1 0 1 active+undersized+degraded 2026-02-23T08:16:52.886117+0000 42'1 44:27 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-23T08:16:47.162192+0000 0'0 2026-02-23T08:16:47.162192+0000 0 0 periodic scrub scheduled @ 2026-02-24T15:07:37.345418+0000 0 0 2026-02-23T08:16:53.715 INFO:tasks.workunit.client.0.trial186.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.947048+0000 0'0 42:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:54:33.056436+0000 0 0 2026-02-23T08:16:53.715 INFO:tasks.workunit.client.0.trial186.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.947188+0000 0'0 44:111 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:20:07.003176+0000 0 0 2026-02-23T08:16:53.715 INFO:tasks.workunit.client.0.trial186.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.733718+0000 0'0 42:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:56:45.414249+0000 0 0 2026-02-23T08:16:53.715 INFO:tasks.workunit.client.0.trial186.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:39.545663+0000 0'0 42:68 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T13:46:10.102710+0000 0 0 2026-02-23T08:16:53.715 INFO:tasks.workunit.client.0.trial186.stdout: 2026-02-23T08:16:53.715 INFO:tasks.workunit.client.0.trial186.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-23T08:16:53.715 INFO:tasks.workunit.client.0.trial186.stderr:dumped pgs 2026-02-23T08:16:53.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-23T08:16:53.725 INFO:tasks.workunit.client.0.trial186.stderr:4+0 records in 2026-02-23T08:16:53.725 INFO:tasks.workunit.client.0.trial186.stderr:4+0 records out 2026-02-23T08:16:53.725 INFO:tasks.workunit.client.0.trial186.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.5883e-05 s, 62.2 MB/s 2026-02-23T08:16:53.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-23T08:16:53.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:53.993 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:53.993 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.441 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.441 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.491 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.566 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.566 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.781 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.781 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:54.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:54.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.387 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.440 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.706 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.706 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:55.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:55.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:56.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:56.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.069 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.069 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:57.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:57.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.408 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.408 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.527 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.527 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.641 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:58.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:58.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.285 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.452 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.452 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.579 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.579 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-23T08:16:59.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:16:59.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.546 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.546 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:00.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-23T08:17:00.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-23T08:17:00.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:17:00.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:17:00.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:17:00.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-23T08:17:00.939 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-23T08:17:00.940 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:17:01.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:17:01.147 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:17:01.147 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:17:01.147 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:17:01.147 INFO:tasks.workunit.client.0.trial186.stderr:2147483647' 2026-02-23T08:17:01.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-23T08:17:01.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-23T08:17:01.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:17:01.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-23T08:17:01.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:17:01.148 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-23T08:17:01.150 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:17:01.151 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:17:01.151 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:17:01.151 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:01.151 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:01.151 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:17:01.151 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:17:01.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:17:01.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:17:01.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:17:01.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-23T08:17:01.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:17:01.214 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:17:01.214 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:17:01.215 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:01.215 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:01.215 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:17:01.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:17:01.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-23T08:17:01.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:17:01.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-23T08:17:01.281 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:17:01.492 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:17:01.492 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr:2147483647' 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:17:01.493 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-23T08:17:01.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:17:01.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:01.496 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:17:01.497 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:17:01.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:17:01.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:17:01.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:17:01.561 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:17:01.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-23T08:17:01.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-23T08:17:01.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:17:01.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:17:01.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:17:01.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:17:01.629 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:17:01.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:17:01.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:17:01.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:17:01.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:17:01.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:17:01.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:17:01.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:17:01.645 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:01.642+0000 7ff7b4af0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:01.646 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:01.642+0000 7ff7b4af0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:01.647 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:01.642+0000 7ff7b4af0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:01.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:17:01.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:17:01.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:17:01.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:17:01.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:17:01.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:01.848 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:01.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:17:01.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:01.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:17:02.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:02.137 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:02.134+0000 7ff7b4af0a40 -1 Falling back to public interface 2026-02-23T08:17:02.780 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:02.778+0000 7ff7b4af0a40 -1 osd.2 42 log_to_monitors true 2026-02-23T08:17:03.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:03.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:03.055 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:17:03.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:17:03.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:03.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:17:03.271 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up out weight 0 up_from 48 up_thru 19 down_at 43 last_clean_interval [19,42) [v2:127.0.0.1:6818/2753006355,v1:127.0.0.1:6819/2753006355] [v2:127.0.0.1:6820/2753006355,v1:127.0.0.1:6821/2753006355] exists,up e82a04ae-5fb3-46a1-b1c6-6707eda6863c 2026-02-23T08:17:03.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:17:03.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:17:03.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:17:03.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-23T08:17:03.514 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already in. 2026-02-23T08:17:03.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-23T08:17:18.526 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-23T08:17:18.527 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-23T08:17:18.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-23T08:17:18.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-23T08:17:18.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-23T08:17:18.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-23T08:17:18.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-23T08:17:18.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-23T08:17:18.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-23T08:17:18.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-23T08:17:18.739 INFO:tasks.workunit.client.0.trial186.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-23T08:17:18.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-23T08:17:18.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-23T08:17:18.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-23T08:17:18.946 INFO:tasks.workunit.client.0.trial186.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-23T08:17:18.946 INFO:tasks.workunit.client.0.trial186.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-23T08:17:04.133471+0000 46'301 52:897 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-23T08:16:47.162192+0000 0'0 2026-02-23T08:16:47.162192+0000 0 0 no scrub is scheduled 0 0 2026-02-23T08:17:18.947 INFO:tasks.workunit.client.0.trial186.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:17:03.575582+0000 0'0 52:60 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T13:08:54.884678+0000 0 0 2026-02-23T08:17:18.947 INFO:tasks.workunit.client.0.trial186.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.947188+0000 0'0 52:127 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:20:07.003176+0000 0 0 2026-02-23T08:17:18.947 INFO:tasks.workunit.client.0.trial186.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.733718+0000 0'0 52:42 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:56:45.414249+0000 0 0 2026-02-23T08:17:18.947 INFO:tasks.workunit.client.0.trial186.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:17:03.575737+0000 0'0 52:106 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T11:45:45.470298+0000 0 0 2026-02-23T08:17:18.947 INFO:tasks.workunit.client.0.trial186.stdout: 2026-02-23T08:17:18.947 INFO:tasks.workunit.client.0.trial186.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-23T08:17:18.948 INFO:tasks.workunit.client.0.trial186.stderr:dumped pgs 2026-02-23T08:17:18.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:17:18.957 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:17:18.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:17:18.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:17:18.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:17:18.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:17:19.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:17:19.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-23T08:17:24.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-23T08:17:24.464 INFO:tasks.workunit.client.0.trial186.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-23T08:17:24.465 INFO:tasks.workunit.client.0.trial186.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-23T08:17:19.118380+0000 46'301 54:909 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-23T08:16:47.162192+0000 0'0 2026-02-23T08:16:47.162192+0000 0 0 no scrub is scheduled 0 0 2026-02-23T08:17:24.465 INFO:tasks.workunit.client.0.trial186.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-23T08:17:19.115951+0000 0'0 54:68 [4,1] 4 [4,1] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T17:02:15.184968+0000 0 0 2026-02-23T08:17:24.465 INFO:tasks.workunit.client.0.trial186.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.947188+0000 0'0 52:127 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:20:07.003176+0000 0 0 2026-02-23T08:17:24.465 INFO:tasks.workunit.client.0.trial186.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:16:46.733718+0000 0'0 54:46 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:56:45.414249+0000 0 0 2026-02-23T08:17:24.465 INFO:tasks.workunit.client.0.trial186.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-23T08:17:19.115919+0000 0'0 54:114 [1,0] 1 [1,0] 1 0'0 2026-02-23T08:16:28.999594+0000 0'0 2026-02-23T08:16:28.999594+0000 0 0 periodic scrub scheduled @ 2026-02-24T17:10:15.575885+0000 0 0 2026-02-23T08:17:24.465 INFO:tasks.workunit.client.0.trial186.stdout: 2026-02-23T08:17:24.465 INFO:tasks.workunit.client.0.trial186.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-23T08:17:24.466 INFO:tasks.workunit.client.0.trial186.stderr:dumped pgs 2026-02-23T08:17:24.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:17:24.560 INFO:tasks.workunit.client.0.trial186.stdout:{ 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "num_missing": 1, 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "num_unfound": 1, 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "objects": [ 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "oid": { 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "oid": "obj250", 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "key": "", 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "snapid": -2, 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "hash": 2249616407, 2026-02-23T08:17:24.561 INFO:tasks.workunit.client.0.trial186.stdout: "max": 0, 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "pool": 2, 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "namespace": "" 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "need": "46'251", 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "have": "0'0", 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "flags": "none", 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "locations": [ 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:17:24.562 INFO:tasks.workunit.client.0.trial186.stdout: ] 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: "state": "NotRecovering", 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: "available_might_have_unfound": true, 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: "might_have_unfound": [ 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "2(4)", 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: "status": "osd is down" 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.563 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.564 INFO:tasks.workunit.client.0.trial186.stdout: "more": false 2026-02-23T08:17:24.564 INFO:tasks.workunit.client.0.trial186.stdout:} 2026-02-23T08:17:24.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-23T08:17:24.652 INFO:tasks.workunit.client.0.trial186.stdout:{ 2026-02-23T08:17:24.652 INFO:tasks.workunit.client.0.trial186.stdout: "snap_trimq": "[]", 2026-02-23T08:17:24.652 INFO:tasks.workunit.client.0.trial186.stdout: "snap_trimq_len": 0, 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: "epoch": 54, 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.653 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: "acting_recovery_backfill": [ 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:17:24.654 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "info": { 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s0", 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "0", 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "46'301", 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "46'250", 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "46'200", 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 301, 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:17:24.655 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "54", 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 40, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 40, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 53, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 40, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:17:24.656 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 53, 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 53, 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 40, 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:17:24.657 INFO:tasks.workunit.client.0.trial186.stdout: "version": "46'301", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 910, 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 54, 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:17:19.118380+0000", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:17:19.118380+0000", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:17:19.118380+0000", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:17:19.118380+0000", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:16:52.276726+0000", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:17:19.116554+0000", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:17:19.116554+0000", 2026-02-23T08:17:24.658 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:17:19.118380+0000", 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:17:19.115476+0000", 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:17:19.115265+0000", 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 53, 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "46'200", 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "46'200", 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "created": 40, 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:17:24.659 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 200, 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:17:24.660 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:17:24.661 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 1232896, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 301, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 1505, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 1, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 1, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 303, 2026-02-23T08:17:24.662 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 1, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 301, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 301, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 1204, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:17:24.663 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:17:24.664 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:17:24.665 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.666 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-23T08:17:24.667 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 300 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),4(2)", 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 1 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.668 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: "peer_info": [ 2026-02-23T08:17:24.669 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "peer": "0(3)", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s3", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "3", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "46'301", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "46'301", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "46'200", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 301, 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-23T08:17:24.670 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 40, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 40, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 53, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 40, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 53, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 53, 2026-02-23T08:17:24.671 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 40, 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "version": "46'301", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 900, 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 53, 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+remapped", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.672 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:16:52.276726+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:16:52.884688+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:16:52.884610+0000", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 53, 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "46'200", 2026-02-23T08:17:24.673 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "46'200", 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "created": 40, 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 200, 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:17:24.674 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-24T15:07:37.345418+0000", 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:17:24.675 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 1232896, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 301, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 1505, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 0, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 0, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 301, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 0, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 301, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:17:24.676 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 301, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 1204, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:17:24.677 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:17:24.678 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:17:24.679 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:17:24.680 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 301 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:17:24.681 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "peer": "1(1)", 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s1", 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "1", 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "46'301", 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "46'301", 2026-02-23T08:17:24.682 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "46'200", 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 301, 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 40, 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 40, 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 53, 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 40, 2026-02-23T08:17:24.683 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 53, 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 53, 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 40, 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "version": "46'301", 2026-02-23T08:17:24.684 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 332, 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 46, 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+remapped", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:16:52.276726+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:16:52.884688+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:16:52.884610+0000", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 53, 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "46'200", 2026-02-23T08:17:24.685 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "46'200", 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "created": 40, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 200, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:17:24.686 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-24T15:07:37.345418+0000", 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 1232896, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 301, 2026-02-23T08:17:24.687 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 1505, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 1, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 301, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 301, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 301, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 1204, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:17:24.688 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:17:24.689 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.690 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.691 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 301 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.692 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "peer": "4(2)", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s2", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "2", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "46'301", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "46'301", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "46'200", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 301, 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:17:24.693 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 40, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 40, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 53, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 40, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 53, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 53, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 40, 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:17:24.694 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "version": "46'301", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 332, 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 46, 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+remapped", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:16:52.276726+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:16:52.886117+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:17:00.934927+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:16:52.884688+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:16:52.884610+0000", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 53, 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "46'200", 2026-02-23T08:17:24.695 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "46'200", 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "created": 40, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 41, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:16:47.162192+0000", 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 200, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:17:24.696 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-24T15:07:37.345418+0000", 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 1232896, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 301, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 1505, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 0, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 301, 2026-02-23T08:17:24.697 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 301, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 301, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 1204, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:17:24.698 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.699 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: 2147483647 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:17:24.700 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 301 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 54, 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:17:24.701 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "recovery_state": [ 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "name": "Started/Primary/Active", 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "enter_time": "2026-02-23T08:17:19.115289+0000", 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "might_have_unfound": [ 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "0(3)", 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "status": "already probed" 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "1(1)", 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "status": "already probed" 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.702 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "2(4)", 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "status": "osd is down" 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "4(2)", 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "status": "already probed" 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "recovery_progress": { 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "backfill_targets": [], 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "waiting_on_backfill": [], 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill_started": "MIN", 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "backfill_info": { 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "begin": "MIN", 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "end": "MIN", 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "objects": [] 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "peer_backfill_info": [], 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "backfills_in_flight": [], 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "recovering": [], 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "pg_backend": { 2026-02-23T08:17:24.703 INFO:tasks.workunit.client.0.trial186.stdout: "recovery_ops": [], 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "read_ops": [] 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "name": "Started", 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "enter_time": "2026-02-23T08:17:19.015041+0000" 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "scrubber": { 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "active": false, 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "must_scrub": false, 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "must_deep_scrub": false, 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "must_repair": false, 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "sched_time": "2026-02-24T15:07:37.345418+0000", 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "schedule": "not registered for scrubbing", 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "urgency": "periodic-regular", 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "shallow-target": { 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s0", 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "level": "shallow", 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "urgency": "periodic-regular", 2026-02-23T08:17:24.704 INFO:tasks.workunit.client.0.trial186.stdout: "sched_time": "2026-02-24T15:07:37.345418+0000", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "orig_sched_time": "2026-02-24T15:07:37.345418+0000", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "last_issue": "ok", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "forced": false, 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "eligible": false, 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "queued": false, 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "active": false 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "deep-target": { 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s0", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "level": "deep", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "urgency": "periodic-regular", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "sched_time": "2026-03-03T20:53:41.597657+0000", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "orig_sched_time": "2026-03-03T20:53:41.597657+0000", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "last_issue": "ok", 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "forced": false, 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "eligible": false, 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "queued": false, 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: "active": false 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:17:24.705 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:17:24.706 INFO:tasks.workunit.client.0.trial186.stdout: "agent_state": {} 2026-02-23T08:17:24.706 INFO:tasks.workunit.client.0.trial186.stdout:} 2026-02-23T08:17:24.706 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:17:24.706 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-23T08:17:24.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:17:24.747 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-23T08:17:24.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-23T08:17:24.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-23T08:17:24.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:17:24.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-23T08:17:24.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-23T08:17:24.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-23T08:17:24.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-23T08:17:24.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:17:24.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:17:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:17:24.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:17:24.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:17:24.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:17:24.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:17:24.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:17:24.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:17:24.997 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:17:24.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:17:24.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:17:24.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:17:24.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:17:24.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:17:24.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:17:24.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:17:25.017 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:25.014+0000 7fa5d5dada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:25.018 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:25.014+0000 7fa5d5dada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:25.019 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:25.014+0000 7fa5d5dada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:25.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:17:25.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:17:25.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:17:25.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:17:25.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:17:25.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:25.204 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:25.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:17:25.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:25.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:17:25.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:25.457 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:25.454+0000 7fa5d5dada40 -1 Falling back to public interface 2026-02-23T08:17:26.017 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:26.014+0000 7fa5d5dada40 -1 osd.2 53 log_to_monitors true 2026-02-23T08:17:26.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:26.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:26.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:17:26.424 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:17:26.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:26.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:17:26.638 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 56 up_thru 19 down_at 53 last_clean_interval [48,53) [v2:127.0.0.1:6818/1117902745,v1:127.0.0.1:6819/1117902745] [v2:127.0.0.1:6820/1117902745,v1:127.0.0.1:6821/1117902745] exists,up e82a04ae-5fb3-46a1-b1c6-6707eda6863c 2026-02-23T08:17:26.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:17:26.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:17:26.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:17:26.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-23T08:17:31.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-23T08:17:31.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-23T08:17:31.717 INFO:tasks.workunit.client.0.trial186.stderr:pg has no unfound objects 2026-02-23T08:17:31.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-23T08:17:31.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:17:31.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:17:31.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:17:31.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:17:31.728 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:17:31.728 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:17:31.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:17:31.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:17:31.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:17:31.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:17:31.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:17:31.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:17:31.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:17:31.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:17:31.839 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:17:32.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:17:32.051 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:17:32.051 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:17:32.051 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:17:32.051 INFO:tasks.workunit.client.0.trial186.stderr:4' 2026-02-23T08:17:32.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:17:32.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:17:32.052 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:17:32.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803792 2026-02-23T08:17:32.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803792 2026-02-23T08:17:32.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792' 2026-02-23T08:17:32.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:17:32.131 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:17:32.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574864 2026-02-23T08:17:32.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574864 2026-02-23T08:17:32.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-55834574864' 2026-02-23T08:17:32.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:17:32.213 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:17:32.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=240518168580 2026-02-23T08:17:32.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 240518168580 2026-02-23T08:17:32.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-55834574864 2-240518168580' 2026-02-23T08:17:32.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:17:32.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:17:32.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117007 2026-02-23T08:17:32.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117007 2026-02-23T08:17:32.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-55834574864 2-240518168580 3-115964117007' 2026-02-23T08:17:32.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:17:32.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:17:32.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855374 2026-02-23T08:17:32.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855374 2026-02-23T08:17:32.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-55834574864 2-240518168580 3-115964117007 4-150323855374' 2026-02-23T08:17:32.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:17:32.453 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803792 2026-02-23T08:17:32.453 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:17:32.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:17:32.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803792 2026-02-23T08:17:32.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:17:32.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803792 2026-02-23T08:17:32.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-23T08:17:32.457 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803792 2026-02-23T08:17:32.457 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:17:32.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-23T08:17:32.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:17:33.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:17:33.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:17:33.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-23T08:17:33.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:17:33.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574864 2026-02-23T08:17:33.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:17:33.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:17:33.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574864 2026-02-23T08:17:33.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:17:33.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574864 2026-02-23T08:17:33.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574864' 2026-02-23T08:17:33.894 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574864 2026-02-23T08:17:33.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:17:34.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-23T08:17:34.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:17:34.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-240518168580 2026-02-23T08:17:34.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:17:34.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:17:34.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-240518168580 2026-02-23T08:17:34.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:17:34.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=240518168580 2026-02-23T08:17:34.113 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 240518168580 2026-02-23T08:17:34.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 240518168580' 2026-02-23T08:17:34.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:17:34.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 240518168580 -lt 240518168580 2026-02-23T08:17:34.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:17:34.342 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117007 2026-02-23T08:17:34.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:17:34.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:17:34.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117007 2026-02-23T08:17:34.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:17:34.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117007 2026-02-23T08:17:34.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117007' 2026-02-23T08:17:34.346 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 115964117007 2026-02-23T08:17:34.347 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:17:34.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117008 -lt 115964117007 2026-02-23T08:17:34.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:17:34.560 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855374 2026-02-23T08:17:34.560 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:17:34.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:17:34.561 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855374 2026-02-23T08:17:34.562 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:17:34.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855374 2026-02-23T08:17:34.563 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 150323855374 2026-02-23T08:17:34.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855374' 2026-02-23T08:17:34.563 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:17:34.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855374 -lt 150323855374 2026-02-23T08:17:34.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:17:34.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:17:34.796 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:17:35.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:17:35.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:17:35.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:17:35.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:17:35.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:17:35.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:17:35.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:17:35.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:17:35.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:17:35.300 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:17:35.301 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:17:35.301 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:17:35.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:17:35.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:17:35.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:17:35.593 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-23T08:17:35.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-23T08:17:35.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.613 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.614 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-23T08:17:35.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-23T08:17:35.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-23T08:17:35.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-23T08:17:35.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-23T08:17:35.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-23T08:17:35.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.752 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-23T08:17:35.752 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-23T08:17:35.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-23T08:17:35.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-23T08:17:35.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-23T08:17:35.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-23T08:17:35.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-23T08:17:35.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-23T08:17:35.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-23T08:17:35.936 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-23T08:17:35.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-23T08:17:35.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:35.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:35.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-23T08:17:35.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-23T08:17:36.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-23T08:17:36.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-23T08:17:36.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-23T08:17:36.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-23T08:17:36.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-23T08:17:36.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-23T08:17:36.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.164 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.164 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-23T08:17:36.164 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-23T08:17:36.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-23T08:17:36.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-23T08:17:36.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-23T08:17:36.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-23T08:17:36.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.304 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.304 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-23T08:17:36.304 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-23T08:17:36.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-23T08:17:36.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-23T08:17:36.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-23T08:17:36.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-23T08:17:36.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.442 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-23T08:17:36.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.464 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-23T08:17:36.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-23T08:17:36.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-23T08:17:36.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-23T08:17:36.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-23T08:17:36.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.579 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.580 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-23T08:17:36.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-23T08:17:36.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-23T08:17:36.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-23T08:17:36.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-23T08:17:36.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-23T08:17:36.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-23T08:17:36.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-23T08:17:36.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-23T08:17:36.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-23T08:17:36.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-23T08:17:36.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-23T08:17:36.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-23T08:17:36.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-23T08:17:36.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-23T08:17:36.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-23T08:17:36.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-23T08:17:36.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-23T08:17:36.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:36.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:36.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-23T08:17:36.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-23T08:17:37.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-23T08:17:37.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.049 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.049 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-23T08:17:37.049 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-23T08:17:37.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.094 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-23T08:17:37.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-23T08:17:37.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-23T08:17:37.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-23T08:17:37.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-23T08:17:37.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-23T08:17:37.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-23T08:17:37.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-23T08:17:37.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-23T08:17:37.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-23T08:17:37.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-23T08:17:37.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-23T08:17:37.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-23T08:17:37.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-23T08:17:37.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-23T08:17:37.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-23T08:17:37.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.442 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-23T08:17:37.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-23T08:17:37.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-23T08:17:37.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-23T08:17:37.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-23T08:17:37.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-23T08:17:37.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-23T08:17:37.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-23T08:17:37.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-23T08:17:37.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-23T08:17:37.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-23T08:17:37.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-23T08:17:37.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-23T08:17:37.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-23T08:17:37.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-23T08:17:37.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-23T08:17:37.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-23T08:17:37.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-23T08:17:37.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-23T08:17:37.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-23T08:17:37.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-23T08:17:37.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-23T08:17:37.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-23T08:17:37.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-23T08:17:37.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-23T08:17:37.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:37.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:37.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-23T08:17:37.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-23T08:17:38.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-23T08:17:38.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.049 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-23T08:17:38.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.069 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-23T08:17:38.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-23T08:17:38.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-23T08:17:38.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-23T08:17:38.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.165 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-23T08:17:38.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-23T08:17:38.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-23T08:17:38.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-23T08:17:38.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-23T08:17:38.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-23T08:17:38.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-23T08:17:38.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-23T08:17:38.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-23T08:17:38.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-23T08:17:38.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-23T08:17:38.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-23T08:17:38.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-23T08:17:38.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.445 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-23T08:17:38.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-23T08:17:38.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-23T08:17:38.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-23T08:17:38.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-23T08:17:38.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-23T08:17:38.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-23T08:17:38.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-23T08:17:38.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-23T08:17:38.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-23T08:17:38.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-23T08:17:38.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-23T08:17:38.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-23T08:17:38.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-23T08:17:38.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-23T08:17:38.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-23T08:17:38.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-23T08:17:38.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.832 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.832 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-23T08:17:38.833 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-23T08:17:38.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-23T08:17:38.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-23T08:17:38.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-23T08:17:38.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-23T08:17:38.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-23T08:17:38.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-23T08:17:38.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:38.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:38.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-23T08:17:38.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-23T08:17:39.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.023 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-23T08:17:39.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-23T08:17:39.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-23T08:17:39.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-23T08:17:39.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-23T08:17:39.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-23T08:17:39.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.158 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-23T08:17:39.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-23T08:17:39.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-23T08:17:39.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-23T08:17:39.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-23T08:17:39.238 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-23T08:17:39.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-23T08:17:39.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-23T08:17:39.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-23T08:17:39.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-23T08:17:39.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-23T08:17:39.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-23T08:17:39.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-23T08:17:39.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-23T08:17:39.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.450 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.450 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-23T08:17:39.450 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-23T08:17:39.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.491 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-23T08:17:39.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-23T08:17:39.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-23T08:17:39.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-23T08:17:39.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-23T08:17:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-23T08:17:39.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-23T08:17:39.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-23T08:17:39.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-23T08:17:39.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-23T08:17:39.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-23T08:17:39.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-23T08:17:39.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-23T08:17:39.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-23T08:17:39.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-23T08:17:39.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-23T08:17:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-23T08:17:39.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-23T08:17:39.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-23T08:17:39.917 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.937 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.937 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-23T08:17:39.937 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.955 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.955 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-23T08:17:39.955 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-23T08:17:39.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:39.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:39.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-23T08:17:39.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-23T08:17:40.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-23T08:17:40.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-23T08:17:40.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-23T08:17:40.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.094 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-23T08:17:40.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-23T08:17:40.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-23T08:17:40.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-23T08:17:40.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-23T08:17:40.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-23T08:17:40.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-23T08:17:40.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-23T08:17:40.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.282 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-23T08:17:40.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-23T08:17:40.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-23T08:17:40.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-23T08:17:40.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-23T08:17:40.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-23T08:17:40.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-23T08:17:40.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-23T08:17:40.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.446 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-23T08:17:40.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-23T08:17:40.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.494 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-23T08:17:40.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-23T08:17:40.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-23T08:17:40.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-23T08:17:40.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.584 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.584 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-23T08:17:40.584 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-23T08:17:40.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-23T08:17:40.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-23T08:17:40.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-23T08:17:40.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-23T08:17:40.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-23T08:17:40.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-23T08:17:40.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-23T08:17:40.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-23T08:17:40.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.794 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-23T08:17:40.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-23T08:17:40.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-23T08:17:40.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-23T08:17:40.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-23T08:17:40.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-23T08:17:40.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-23T08:17:40.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-23T08:17:40.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-23T08:17:40.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:40.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:40.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-23T08:17:40.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-23T08:17:41.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.023 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-23T08:17:41.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-23T08:17:41.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.069 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.070 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-23T08:17:41.070 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-23T08:17:41.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-23T08:17:41.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-23T08:17:41.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-23T08:17:41.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-23T08:17:41.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.199 INFO:tasks.workunit.client.0.trial186.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-23T08:17:41.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-23T08:17:41.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-23T08:17:41.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-23T08:17:41.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-23T08:17:41.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-23T08:17:41.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-23T08:17:41.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-23T08:17:41.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-23T08:17:41.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-23T08:17:41.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-23T08:17:41.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.435 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-23T08:17:41.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-23T08:17:41.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-23T08:17:41.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-23T08:17:41.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-23T08:17:41.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-23T08:17:41.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-23T08:17:41.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-23T08:17:41.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-23T08:17:41.612 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-23T08:17:41.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-23T08:17:41.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-23T08:17:41.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-23T08:17:41.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-23T08:17:41.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-23T08:17:41.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-23T08:17:41.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-23T08:17:41.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-23T08:17:41.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.828 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-23T08:17:41.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-23T08:17:41.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-23T08:17:41.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-23T08:17:41.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-23T08:17:41.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-23T08:17:41.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-23T08:17:41.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-23T08:17:41.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:41.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:41.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-23T08:17:41.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-23T08:17:42.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-23T08:17:42.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-23T08:17:42.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-23T08:17:42.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-23T08:17:42.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-23T08:17:42.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-23T08:17:42.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-23T08:17:42.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-23T08:17:42.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-23T08:17:42.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-23T08:17:42.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-23T08:17:42.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-23T08:17:42.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.284 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.285 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:17:42.285 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-23T08:17:42.285 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-23T08:17:42.310 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:17:42.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:17:42.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:17:42.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:17:42.566 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:17:42.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:17:42.850 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:17:42.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:17:42.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:17:42.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:17:42.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:17:42.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:17:42.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:17:42.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:17:42.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:17:42.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:17:42.972 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:17:42.972 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:17:42.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:17:42.973 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:17:42.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:17:42.975 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:17:42.975 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:17:42.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:17:42.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:17:42.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:17:42.977 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:17:42.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:17:42.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:17:42.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:17:43.022 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:17:43.022 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:43.022 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:43.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:17:43.024 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:17:43.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:17:43.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:17:43.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:17:43.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:17:43.028 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:17:43.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:17:43.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:17:43.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:17:43.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:17:43.030 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:17:43.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:17:43.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:17:43.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:17:43.032 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:17:43.033 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:17:43.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:17:43.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:17:43.036 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:17:43.036 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:43.036 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:43.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:17:43.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:17:43.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:17:43.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:17:43.039 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:17:43.039 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:43.039 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:43.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:17:43.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:17:43.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:17:43.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:17:43.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:43.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:43.064 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:43.064 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:43.064 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:43.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:43.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:17:43.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:17:43.093 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:17:43.093 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:17:43.093 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:17:43.093 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:17:43.094 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:17:43.094 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:17:43.094 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:17:43.094 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:17:43.094 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:17:43.095 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:43.095 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:43.095 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:17:43.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:17:43.095 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:17:43.153 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:17:43.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:17:43.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:17:43.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:17:43.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:17:43.154 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:17:43.154 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:17:43.154 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:17:43.156 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:17:43.156 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:43.156 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:43.156 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:17:43.156 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:17:43.156 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:17:43.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:17:43.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:17:43.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:17:43.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:17:43.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:17:43.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:17:43.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:17:43.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:17:43.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:43.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:43.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:43.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:43.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:43.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:43.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:17:43.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:17:43.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:17:43.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:17:43.456 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:17:43.464 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:17:44.467 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:17:44.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:17:44.516 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:17:44.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:17:44.523 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:17:43.078+0000 7f2d4888bec0 0 load: jerasure load: lrc 2026-02-23T08:17:44.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-23T08:17:44.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-23T08:17:44.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-23T08:17:44.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-23T08:17:44.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-23T08:17:44.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:17:44.525 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-23T08:17:44.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-23T08:17:44.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:17:44.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:17:44.529 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:44.530 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:17:44.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:17:44.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:17:44.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:17:44.532 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:17:44.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0acc1b53-6b64-4b8e-badd-71e5b0a850c6 2026-02-23T08:17:44.533 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0acc1b53-6b64-4b8e-badd-71e5b0a850c6' 2026-02-23T08:17:44.533 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 0acc1b53-6b64-4b8e-badd-71e5b0a850c6 2026-02-23T08:17:44.533 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:17:44.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAoDZxpeGQpIBAAHvJAcl3QVpt8HTwIGH45KQ== 2026-02-23T08:17:44.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAoDZxpeGQpIBAAHvJAcl3QVpt8HTwIGH45KQ=="}' 2026-02-23T08:17:44.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0acc1b53-6b64-4b8e-badd-71e5b0a850c6 -i td/test-erasure-eio/0/new.json 2026-02-23T08:17:44.654 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:44.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:17:44.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAoDZxpeGQpIBAAHvJAcl3QVpt8HTwIGH45KQ== --osd-uuid 0acc1b53-6b64-4b8e-badd-71e5b0a850c6 2026-02-23T08:17:44.679 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:44.674+0000 7f188a00ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:44.680 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:44.678+0000 7f188a00ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:44.681 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:44.678+0000 7f188a00ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:44.682 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:44.678+0000 7f188a00ca40 -1 bdev(0x557af0e0e400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:17:44.682 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:44.678+0000 7f188a00ca40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:17:45.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:17:45.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:17:45.363 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:17:45.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:17:45.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:17:45.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:17:45.474 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:17:45.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:17:45.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:17:45.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:17:45.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:17:45.494 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:45.490+0000 7f5bf3653a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:45.496 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:45.490+0000 7f5bf3653a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:45.498 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:45.494+0000 7f5bf3653a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:45.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:17:45.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:17:45.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:17:45.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:17:45.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:17:45.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:45.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:17:45.610 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:45.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:45.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:17:45.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:45.923 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:45.918+0000 7f5bf3653a40 -1 Falling back to public interface 2026-02-23T08:17:46.469 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:46.466+0000 7f5bf3653a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:17:46.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:46.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:46.723 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:17:46.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:17:46.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:46.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:17:46.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:47.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:47.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:47.949 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:17:47.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:17:47.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:47.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:17:48.167 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/403310692,v1:127.0.0.1:6803/403310692] [v2:127.0.0.1:6804/403310692,v1:127.0.0.1:6805/403310692] exists,up 0acc1b53-6b64-4b8e-badd-71e5b0a850c6 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:17:48.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:48.172 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:17:48.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:17:48.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:17:48.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:17:48.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:17:48.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:17:48.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:17:48.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8ed30341-38bd-4822-a53f-20d5f5183131 2026-02-23T08:17:48.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 8ed30341-38bd-4822-a53f-20d5f5183131' 2026-02-23T08:17:48.175 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 8ed30341-38bd-4822-a53f-20d5f5183131 2026-02-23T08:17:48.175 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:17:48.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAsDZxpGEbgChAAYDAQvJeInG1LLpGU7/dbaA== 2026-02-23T08:17:48.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAsDZxpGEbgChAAYDAQvJeInG1LLpGU7/dbaA=="}' 2026-02-23T08:17:48.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8ed30341-38bd-4822-a53f-20d5f5183131 -i td/test-erasure-eio/1/new.json 2026-02-23T08:17:48.409 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:17:48.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:17:48.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAsDZxpGEbgChAAYDAQvJeInG1LLpGU7/dbaA== --osd-uuid 8ed30341-38bd-4822-a53f-20d5f5183131 2026-02-23T08:17:48.437 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:48.433+0000 7f63cc33fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:48.439 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:48.433+0000 7f63cc33fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:48.440 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:48.433+0000 7f63cc33fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:48.440 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:48.437+0000 7f63cc33fa40 -1 bdev(0x55fe0522a400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:17:48.441 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:48.437+0000 7f63cc33fa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:17:49.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:17:49.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:17:49.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:17:49.049 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:17:49.049 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:17:49.356 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:17:49.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:17:49.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:17:49.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:17:49.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:17:49.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:17:49.380 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:49.377+0000 7fc106503a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:49.382 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:49.377+0000 7fc106503a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:49.383 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:49.377+0000 7fc106503a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:49.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:17:49.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:17:49.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:17:49.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:17:49.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:17:49.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:49.569 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:49.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:17:49.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:49.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:17:49.734 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:49.729+0000 7fc106503a40 -1 Falling back to public interface 2026-02-23T08:17:49.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:50.354 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:50.349+0000 7fc106503a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:17:50.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:50.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:50.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:17:50.793 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:17:50.793 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:50.793 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:17:51.003 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1837521160,v1:127.0.0.1:6811/1837521160] [v2:127.0.0.1:6812/1837521160,v1:127.0.0.1:6813/1837521160] exists,up 8ed30341-38bd-4822-a53f-20d5f5183131 2026-02-23T08:17:51.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:17:51.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:17:51.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:17:51.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:17:51.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:17:51.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:17:51.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:17:51.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:17:51.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9ed97340-e130-43ab-908f-644232e6e2c2 2026-02-23T08:17:51.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 9ed97340-e130-43ab-908f-644232e6e2c2' 2026-02-23T08:17:51.013 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 9ed97340-e130-43ab-908f-644232e6e2c2 2026-02-23T08:17:51.013 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:17:51.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAvDZxpFFkbARAAaweXbw1rTJY1tUJSYlBBAQ== 2026-02-23T08:17:51.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAvDZxpFFkbARAAaweXbw1rTJY1tUJSYlBBAQ=="}' 2026-02-23T08:17:51.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9ed97340-e130-43ab-908f-644232e6e2c2 -i td/test-erasure-eio/2/new.json 2026-02-23T08:17:51.242 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:17:51.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:17:51.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAvDZxpFFkbARAAaweXbw1rTJY1tUJSYlBBAQ== --osd-uuid 9ed97340-e130-43ab-908f-644232e6e2c2 2026-02-23T08:17:51.269 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:51.265+0000 7fd45e3eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:51.270 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:51.265+0000 7fd45e3eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:51.271 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:51.265+0000 7fd45e3eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:51.272 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:51.265+0000 7fd45e3eba40 -1 bdev(0x558043e30400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:17:51.272 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:51.265+0000 7fd45e3eba40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:17:51.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:17:51.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:17:51.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:17:51.913 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:17:51.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:17:52.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:17:52.203 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:17:52.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:17:52.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:17:52.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:17:52.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:17:52.225 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:52.221+0000 7f1a41078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:52.226 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:52.221+0000 7f1a41078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:52.227 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:52.221+0000 7f1a41078a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:52.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:17:52.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:17:52.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:17:52.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:17:52.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:17:52.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:52.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:17:52.423 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:52.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:52.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:17:52.610 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:52.605+0000 7f1a41078a40 -1 Falling back to public interface 2026-02-23T08:17:52.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:53.205 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:53.201+0000 7f1a41078a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:17:53.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:53.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:53.633 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:17:53.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:17:53.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:53.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:17:53.857 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1777899502,v1:127.0.0.1:6819/1777899502] [v2:127.0.0.1:6820/1777899502,v1:127.0.0.1:6821/1777899502] exists,up 9ed97340-e130-43ab-908f-644232e6e2c2 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:17:53.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:53.862 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:17:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:17:53.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:17:53.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:17:53.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:17:53.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:17:53.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:17:53.864 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:17:53.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d8fe6d56-17b4-4200-91ea-df44a52569ce 2026-02-23T08:17:53.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 d8fe6d56-17b4-4200-91ea-df44a52569ce' 2026-02-23T08:17:53.865 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 d8fe6d56-17b4-4200-91ea-df44a52569ce 2026-02-23T08:17:53.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:17:53.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAxDZxpRcUTNBAAgscPi/R8ATc86TFUDTCugQ== 2026-02-23T08:17:53.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAxDZxpRcUTNBAAgscPi/R8ATc86TFUDTCugQ=="}' 2026-02-23T08:17:53.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d8fe6d56-17b4-4200-91ea-df44a52569ce -i td/test-erasure-eio/3/new.json 2026-02-23T08:17:54.106 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:17:54.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:17:54.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAxDZxpRcUTNBAAgscPi/R8ATc86TFUDTCugQ== --osd-uuid d8fe6d56-17b4-4200-91ea-df44a52569ce 2026-02-23T08:17:54.142 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:54.137+0000 7f6676763a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:54.144 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:54.137+0000 7f6676763a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:54.145 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:54.141+0000 7f6676763a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:54.146 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:54.141+0000 7f6676763a40 -1 bdev(0x55bb2972a400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:17:54.146 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:54.141+0000 7f6676763a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:17:54.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:17:54.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:17:54.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:17:54.813 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:17:54.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:17:55.103 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:17:55.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:17:55.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:17:55.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:17:55.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:17:55.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:17:55.127 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:55.121+0000 7f956ce84a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:55.128 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:55.125+0000 7f956ce84a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:55.129 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:55.125+0000 7f956ce84a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:55.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:17:55.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:17:55.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:17:55.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:17:55.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:17:55.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:55.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:17:55.329 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:55.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:55.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:17:55.478 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:55.473+0000 7f956ce84a40 -1 Falling back to public interface 2026-02-23T08:17:55.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:56.126 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:56.121+0000 7f956ce84a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:17:56.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:56.544 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:17:56.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:56.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:17:56.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:56.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:17:56.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:57.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:17:57.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:57.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:17:57.766 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:17:57.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:57.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:17:57.984 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 26 up_thru 29 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3392716348,v1:127.0.0.1:6827/3392716348] [v2:127.0.0.1:6828/3392716348,v1:127.0.0.1:6829/3392716348] exists,up d8fe6d56-17b4-4200-91ea-df44a52569ce 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:17:57.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:17:57.989 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:17:57.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=16288377-d9d1-484c-80df-dcf2716ee1c1 2026-02-23T08:17:57.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 16288377-d9d1-484c-80df-dcf2716ee1c1' 2026-02-23T08:17:57.992 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 16288377-d9d1-484c-80df-dcf2716ee1c1 2026-02-23T08:17:57.992 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:17:58.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA1DZxpd5KBOxAAZq9tgBKkwL+0Kk6dryjXPg== 2026-02-23T08:17:58.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA1DZxpd5KBOxAAZq9tgBKkwL+0Kk6dryjXPg=="}' 2026-02-23T08:17:58.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 16288377-d9d1-484c-80df-dcf2716ee1c1 -i td/test-erasure-eio/4/new.json 2026-02-23T08:17:58.225 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:17:58.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-23T08:17:58.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA1DZxpd5KBOxAAZq9tgBKkwL+0Kk6dryjXPg== --osd-uuid 16288377-d9d1-484c-80df-dcf2716ee1c1 2026-02-23T08:17:58.253 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:58.249+0000 7f4e80000a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:58.254 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:58.249+0000 7f4e80000a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:58.255 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:58.249+0000 7f4e80000a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:58.257 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:58.249+0000 7f4e80000a40 -1 bdev(0x557ef15c8400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:17:58.257 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:58.249+0000 7f4e80000a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-23T08:17:58.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-23T08:17:58.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:17:58.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:17:58.913 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:17:58.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:17:59.215 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:17:59.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:17:59.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:17:59.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:17:59.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:17:59.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:17:59.239 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:59.233+0000 7f8029efaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:59.241 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:59.237+0000 7f8029efaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:59.242 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:59.237+0000 7f8029efaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:17:59.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:17:59.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:17:59.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:17:59.682 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:17:59.677+0000 7f8029efaa40 -1 Falling back to public interface 2026-02-23T08:18:00.310 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:00.305+0000 7f8029efaa40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:18:00.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:00.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:00.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:00.651 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:00.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:00.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:18:00.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:01.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:01.892 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:01.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:01.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:18:01.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:01.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:18:02.104 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 36 up_thru 37 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2695795214,v1:127.0.0.1:6835/2695795214] [v2:127.0.0.1:6836/2695795214,v1:127.0.0.1:6837/2695795214] exists,up 16288377-d9d1-484c-80df-dcf2716ee1c1 2026-02-23T08:18:02.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:02.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:02.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:02.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:02.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-23T08:18:02.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:02.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:02.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:02.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:02.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:02.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:02.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:02.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-23T08:18:02.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:02.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a18af21d-965e-4c7d-bfb6-9c5c16351f44 2026-02-23T08:18:02.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 a18af21d-965e-4c7d-bfb6-9c5c16351f44' 2026-02-23T08:18:02.112 INFO:tasks.workunit.client.0.trial186.stdout:add osd5 a18af21d-965e-4c7d-bfb6-9c5c16351f44 2026-02-23T08:18:02.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:02.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA6DZxp+TsxBxAAAYzGFlnBdKs9WPe+KPR8PQ== 2026-02-23T08:18:02.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA6DZxp+TsxBxAAAYzGFlnBdKs9WPe+KPR8PQ=="}' 2026-02-23T08:18:02.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a18af21d-965e-4c7d-bfb6-9c5c16351f44 -i td/test-erasure-eio/5/new.json 2026-02-23T08:18:02.341 INFO:tasks.workunit.client.0.trial186.stdout:5 2026-02-23T08:18:02.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-23T08:18:02.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA6DZxp+TsxBxAAAYzGFlnBdKs9WPe+KPR8PQ== --osd-uuid a18af21d-965e-4c7d-bfb6-9c5c16351f44 2026-02-23T08:18:02.375 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:02.369+0000 7f80698c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:02.377 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:02.373+0000 7f80698c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:02.378 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:02.373+0000 7f80698c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:02.379 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:02.373+0000 7f80698c0a40 -1 bdev(0x558434ba0400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:02.379 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:02.373+0000 7f80698c0a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-23T08:18:03.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-23T08:18:03.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:03.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-23T08:18:03.005 INFO:tasks.workunit.client.0.trial186.stdout:adding osd5 key to auth repository 2026-02-23T08:18:03.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:03.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-23T08:18:03.305 INFO:tasks.workunit.client.0.trial186.stdout:start osd.5 2026-02-23T08:18:03.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:03.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:03.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:03.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:03.328 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:03.321+0000 7f9d9763ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:03.330 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:03.325+0000 7f9d9763ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:03.331 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:03.325+0000 7f9d9763ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:03.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-23T08:18:03.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:03.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-23T08:18:03.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:03.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:03.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:03.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:03.519 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:03.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:03.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:18:03.658 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:03.653+0000 7f9d9763ea40 -1 Falling back to public interface 2026-02-23T08:18:03.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:04.307 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:04.301+0000 7f9d9763ea40 -1 osd.5 0 log_to_monitors true 2026-02-23T08:18:04.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:04.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:04.739 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:04.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:04.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:04.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:18:04.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:05.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:05.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:05.969 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:05.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:18:05.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:05.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:18:06.188 INFO:tasks.workunit.client.0.trial186.stdout:osd.5 up in weight 1 up_from 44 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/10270930,v1:127.0.0.1:6843/10270930] [v2:127.0.0.1:6844/10270930,v1:127.0.0.1:6845/10270930] exists,up a18af21d-965e-4c7d-bfb6-9c5c16351f44 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-23T08:18:06.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:06.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:06.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:06.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:06.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:06.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:06.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:06.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-23T08:18:06.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:06.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a8ea0eb4-23ff-4929-acb3-10b191d0b031 2026-02-23T08:18:06.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 a8ea0eb4-23ff-4929-acb3-10b191d0b031' 2026-02-23T08:18:06.196 INFO:tasks.workunit.client.0.trial186.stdout:add osd6 a8ea0eb4-23ff-4929-acb3-10b191d0b031 2026-02-23T08:18:06.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:06.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+DZxp0k4KDBAAXszYIWh+W7ffZ9qFlfqJ8w== 2026-02-23T08:18:06.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+DZxp0k4KDBAAXszYIWh+W7ffZ9qFlfqJ8w=="}' 2026-02-23T08:18:06.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a8ea0eb4-23ff-4929-acb3-10b191d0b031 -i td/test-erasure-eio/6/new.json 2026-02-23T08:18:06.423 INFO:tasks.workunit.client.0.trial186.stdout:6 2026-02-23T08:18:06.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-23T08:18:06.435 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA+DZxp0k4KDBAAXszYIWh+W7ffZ9qFlfqJ8w== --osd-uuid a8ea0eb4-23ff-4929-acb3-10b191d0b031 2026-02-23T08:18:06.459 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:06.453+0000 7fcb326dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:06.460 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:06.453+0000 7fcb326dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:06.461 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:06.457+0000 7fcb326dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:06.462 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:06.457+0000 7fcb326dfa40 -1 bdev(0x55d4be014400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:06.462 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:06.457+0000 7fcb326dfa40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-23T08:18:07.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-23T08:18:07.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:07.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-23T08:18:07.145 INFO:tasks.workunit.client.0.trial186.stdout:adding osd6 key to auth repository 2026-02-23T08:18:07.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:07.447 INFO:tasks.workunit.client.0.trial186.stdout:start osd.6 2026-02-23T08:18:07.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-23T08:18:07.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:07.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:07.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:07.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:07.470 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:07.465+0000 7f36dd1d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:07.472 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:07.465+0000 7f36dd1d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:07.473 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:07.469+0000 7f36dd1d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:07.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-23T08:18:07.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:07.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-23T08:18:07.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:07.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:07.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:07.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:07.668 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:07.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:07.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:18:07.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:07.910 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:07.905+0000 7f36dd1d4a40 -1 Falling back to public interface 2026-02-23T08:18:08.550 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:08.545+0000 7f36dd1d4a40 -1 osd.6 0 log_to_monitors true 2026-02-23T08:18:08.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:08.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:08.885 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:08.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:08.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:08.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:18:09.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:10.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:10.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:10.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:18:10.108 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:10.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:10.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:18:10.321 INFO:tasks.workunit.client.0.trial186.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3962201386,v1:127.0.0.1:6851/3962201386] [v2:127.0.0.1:6852/3962201386,v1:127.0.0.1:6853/3962201386] exists,up a8ea0eb4-23ff-4929-acb3-10b191d0b031 2026-02-23T08:18:10.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:10.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:10.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:10.322 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:18:10.322 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:18:10.322 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:18:10.323 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:18:10.323 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:10.323 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:10.323 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:18:10.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:18:10.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:18:10.389 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:18:10.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:18:10.397 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:17:45.922+0000 7f5bf3653a40 0 load: jerasure load: lrc 2026-02-23T08:18:10.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:18:10.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-23T08:18:10.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:18:10.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:18:11.064 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:18:11.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:18:12.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:18:12.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:18:12.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:18:12.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:18:12.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:18:12.076 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:18:12.076 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:18:12.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:18:12.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:18:12.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:18:12.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:18:12.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:18:12.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:18:12.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:18:12.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:18:12.168 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:18:12.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:18:12.382 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:18:12.382 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:18:12.382 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:18:12.382 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:18:12.382 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:18:12.382 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:18:12.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:18:12.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:12.383 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:18:12.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-23T08:18:12.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-23T08:18:12.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783' 2026-02-23T08:18:12.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:12.463 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:18:12.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-23T08:18:12.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-23T08:18:12.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-51539607559' 2026-02-23T08:18:12.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:12.541 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:18:12.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-23T08:18:12.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-23T08:18:12.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334' 2026-02-23T08:18:12.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:12.622 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:18:12.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-23T08:18:12.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-23T08:18:12.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702' 2026-02-23T08:18:12.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:12.702 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:18:12.781 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822661 2026-02-23T08:18:12.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822661 2026-02-23T08:18:12.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-154618822661' 2026-02-23T08:18:12.782 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:12.782 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:18:12.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-23T08:18:12.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-23T08:18:12.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-154618822661 5-188978561028' 2026-02-23T08:18:12.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:12.858 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:18:12.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332099 2026-02-23T08:18:12.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332099 2026-02-23T08:18:12.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-154618822661 5-188978561028 6-219043332099' 2026-02-23T08:18:12.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:12.938 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803783 2026-02-23T08:18:12.938 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:12.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:18:12.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-23T08:18:12.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:12.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803783 2026-02-23T08:18:12.942 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803783 2026-02-23T08:18:12.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-23T08:18:12.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:18:13.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-23T08:18:13.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:18:14.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:18:14.153 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:18:14.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803783 2026-02-23T08:18:14.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:14.372 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607559 2026-02-23T08:18:14.372 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:14.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:18:14.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-23T08:18:14.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:14.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607559 2026-02-23T08:18:14.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-23T08:18:14.376 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607559 2026-02-23T08:18:14.376 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:18:14.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-23T08:18:14.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:14.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411334 2026-02-23T08:18:14.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:14.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:18:14.602 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411334 2026-02-23T08:18:14.602 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:14.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-23T08:18:14.604 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411334 2026-02-23T08:18:14.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-23T08:18:14.605 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:18:14.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-23T08:18:14.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:14.826 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-23T08:18:14.826 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:14.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:18:14.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-23T08:18:14.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:14.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-23T08:18:14.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-23T08:18:14.829 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 111669149702 2026-02-23T08:18:14.830 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:18:15.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-23T08:18:15.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:15.047 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822661 2026-02-23T08:18:15.047 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:15.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:18:15.049 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822661 2026-02-23T08:18:15.049 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:15.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822661 2026-02-23T08:18:15.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822661' 2026-02-23T08:18:15.051 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 154618822661 2026-02-23T08:18:15.051 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:18:15.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822661 -lt 154618822661 2026-02-23T08:18:15.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:15.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-188978561028 2026-02-23T08:18:15.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:15.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:18:15.278 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561028 2026-02-23T08:18:15.279 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:15.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561028 2026-02-23T08:18:15.280 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 188978561028 2026-02-23T08:18:15.281 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-23T08:18:15.281 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:18:15.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-23T08:18:15.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:15.499 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332099 2026-02-23T08:18:15.499 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:15.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:18:15.501 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332099 2026-02-23T08:18:15.501 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:15.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332099 2026-02-23T08:18:15.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332099' 2026-02-23T08:18:15.503 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 219043332099 2026-02-23T08:18:15.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:18:15.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332100 -lt 219043332099 2026-02-23T08:18:15.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:18:15.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:18:15.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:18:16.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:18:16.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:18:16.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:18:16.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:18:16.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:18:16.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:18:16.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:18:16.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:18:16.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:18:16.209 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:18:16.209 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:18:16.210 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:18:16.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:18:16.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:18:16.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:18:16.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-23T08:18:16.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-23T08:18:16.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-23T08:18:16.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:18:16.486 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-23T08:18:16.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-23T08:18:16.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-23T08:18:16.704 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:18:16.704 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:18:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:18:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:18:16.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:18:17.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:18:17.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-23T08:18:17.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-23T08:18:17.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:18:17.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:18:17.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=mdata 2026-02-23T08:18:17.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:18:17.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:18:17.619 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:18:17.620 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=5 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:18:17.836 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/5/type 2026-02-23T08:18:17.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:18:17.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-23T08:18:17.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:18:17.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-23T08:18:17.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:18:17.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:18:17.838 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:18:17.838 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-23T08:18:17.838 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-23T08:18:17.839 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-23T08:18:17.839 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:18:17.839 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:17.839 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:17.839 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.5.asok 2026-02-23T08:18:17.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:18:17.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:18:17.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:18:17.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:18:17.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.5 2026-02-23T08:18:17.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-23T08:18:17.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-23T08:18:17.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:18:17.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:17.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:17.903 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.5.asok 2026-02-23T08:18:17.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:18:17.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-23T08:18:17.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-23T08:18:17.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:17.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:18:17.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-23T08:18:17.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:18:17.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:17.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:17.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:17.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:17.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:17.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:18:17.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:18:17.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:18:17.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-23T08:18:17.970 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:18:17.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:17.971 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-23T08:18:17.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-23T08:18:17.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:18:17.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:18:17.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:18:17.986 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:17.981+0000 7f716a7a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:17.987 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:17.981+0000 7f716a7a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:17.988 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:17.981+0000 7f716a7a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:18.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:18:18.318 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:18.313+0000 7f716a7a1a40 -1 Falling back to public interface 2026-02-23T08:18:18.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:18.984 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:18.977+0000 7f716a7a1a40 -1 osd.3 59 log_to_monitors true 2026-02-23T08:18:19.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:19.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:19.404 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:19.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:19.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:19.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:18:19.622 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [26,59) [v2:127.0.0.1:6826/4118653690,v1:127.0.0.1:6827/4118653690] [v2:127.0.0.1:6828/4118653690,v1:127.0.0.1:6829/4118653690] exists,up d8fe6d56-17b4-4200-91ea-df44a52569ce 2026-02-23T08:18:19.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:19.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:19.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:19.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:18:19.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:18:19.731 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:18:19.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:18:19.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:18:19.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:18:19.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:18:19.732 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:19.950 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:18:20.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-23T08:18:20.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-23T08:18:20.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-23T08:18:20.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:20.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:18:20.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607562 2026-02-23T08:18:20.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607562 2026-02-23T08:18:20.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607562' 2026-02-23T08:18:20.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:20.115 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:18:20.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411338 2026-02-23T08:18:20.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411338 2026-02-23T08:18:20.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338' 2026-02-23T08:18:20.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:20.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:18:20.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939651 2026-02-23T08:18:20.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939651 2026-02-23T08:18:20.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-270582939651' 2026-02-23T08:18:20.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:20.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:18:20.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822664 2026-02-23T08:18:20.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822664 2026-02-23T08:18:20.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-270582939651 4-154618822664' 2026-02-23T08:18:20.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:20.356 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:18:20.435 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561031 2026-02-23T08:18:20.435 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561031 2026-02-23T08:18:20.435 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-270582939651 4-154618822664 5-188978561031' 2026-02-23T08:18:20.435 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:20.436 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:18:20.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332103 2026-02-23T08:18:20.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332103 2026-02-23T08:18:20.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-270582939651 4-154618822664 5-188978561031 6-219043332103' 2026-02-23T08:18:20.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:20.515 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-23T08:18:20.515 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:20.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:18:20.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-23T08:18:20.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:20.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-23T08:18:20.518 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803787 2026-02-23T08:18:20.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-23T08:18:20.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:18:20.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-23T08:18:20.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:18:21.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:18:21.747 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:18:21.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-23T08:18:21.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:21.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607562 2026-02-23T08:18:21.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:21.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:18:21.966 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607562 2026-02-23T08:18:21.966 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:21.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607562 2026-02-23T08:18:21.968 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607562 2026-02-23T08:18:21.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607562' 2026-02-23T08:18:21.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:18:22.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607563 -lt 51539607562 2026-02-23T08:18:22.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:22.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411338 2026-02-23T08:18:22.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:22.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:18:22.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411338 2026-02-23T08:18:22.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:22.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411338 2026-02-23T08:18:22.197 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411338 2026-02-23T08:18:22.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411338' 2026-02-23T08:18:22.197 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:18:22.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-23T08:18:22.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:22.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-270582939651 2026-02-23T08:18:22.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:22.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:18:22.418 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-270582939651 2026-02-23T08:18:22.418 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:22.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=270582939651 2026-02-23T08:18:22.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 270582939651' 2026-02-23T08:18:22.420 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 270582939651 2026-02-23T08:18:22.421 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:18:22.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-23T08:18:22.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:22.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822664 2026-02-23T08:18:22.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:22.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:18:22.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822664 2026-02-23T08:18:22.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:22.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822664 2026-02-23T08:18:22.651 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 154618822664 2026-02-23T08:18:22.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822664' 2026-02-23T08:18:22.651 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:18:22.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822665 -lt 154618822664 2026-02-23T08:18:22.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:22.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-188978561031 2026-02-23T08:18:22.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:22.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:18:22.875 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561031 2026-02-23T08:18:22.875 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:22.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561031 2026-02-23T08:18:22.876 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 188978561031 2026-02-23T08:18:22.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 188978561031' 2026-02-23T08:18:22.877 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:18:23.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561032 -lt 188978561031 2026-02-23T08:18:23.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:23.107 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332103 2026-02-23T08:18:23.107 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:23.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:18:23.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332103 2026-02-23T08:18:23.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:23.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332103 2026-02-23T08:18:23.111 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 219043332103 2026-02-23T08:18:23.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332103' 2026-02-23T08:18:23.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:18:23.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332103 -lt 219043332103 2026-02-23T08:18:23.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:18:23.331 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:18:23.331 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:18:23.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:18:23.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:18:23.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:18:23.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:18:23.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:18:23.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:18:23.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:18:23.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:18:23.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:18:23.837 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:18:23.837 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:18:23.837 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:18:24.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:18:24.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:18:24.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:18:24.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:18:24.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:18:24.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:18:24.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-23T08:18:24.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:18:24.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:18:24.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-23T08:18:24.149 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:18:24.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:18:24.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:18:24.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:18:24.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:18:24.419 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:18:24.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:18:24.696 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:18:24.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:18:24.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:18:24.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:18:24.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:18:24.709 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:18:24.709 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:18:24.709 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:18:24.709 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:18:24.709 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:18:24.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:18:24.820 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:18:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:18:24.821 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:18:24.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:18:24.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:18:24.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:18:24.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:18:24.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:18:24.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:18:24.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:18:24.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:18:24.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:18:24.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:18:24.848 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:18:24.848 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:24.848 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:24.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:18:24.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:18:24.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:18:24.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:18:24.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:18:24.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:18:24.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:18:24.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:18:24.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:18:24.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:18:24.854 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:18:24.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:18:24.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:18:24.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:18:24.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:18:24.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:18:24.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:18:24.857 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:18:24.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:18:24.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:18:24.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:18:24.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:18:24.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:24.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:24.861 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:18:24.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:18:24.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:18:24.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:18:24.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:18:24.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:24.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:24.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:18:24.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:18:24.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:18:24.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:18:24.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:18:24.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:18:24.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:18:24.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:18:24.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:18:24.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:18:24.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:18:24.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:18:24.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:18:24.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:24.887 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:24.888 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:24.888 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:24.888 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:24.888 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:24.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:18:24.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:18:24.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:18:24.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:18:24.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:18:24.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:18:24.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:18:24.921 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:18:24.922 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:18:24.922 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:18:24.922 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:18:24.922 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:24.922 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:24.923 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:18:24.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:18:24.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:18:24.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:18:24.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:18:24.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:18:24.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:18:24.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:18:24.978 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:18:24.978 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:18:24.979 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:18:24.979 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:18:24.980 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:24.980 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:24.980 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:18:24.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:18:24.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:18:25.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:18:25.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:18:25.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:18:25.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:18:25.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:18:25.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:18:25.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:18:25.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:18:25.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:25.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:25.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:25.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:25.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:25.144 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:25.144 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:18:25.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:18:25.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:18:25.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:18:25.279 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:18:25.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:18:26.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:18:26.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:18:26.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:18:26.290 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:18:26.290 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:26.290 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:26.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:18:26.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:18:26.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:18:26.340 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:18:26.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:18:26.348 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:18:24.905+0000 7f393c24cec0 0 load: jerasure load: lrc 2026-02-23T08:18:26.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-23T08:18:26.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-23T08:18:26.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-23T08:18:26.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-23T08:18:26.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-23T08:18:26.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:18:26.349 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-23T08:18:26.350 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:26.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:26.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:26.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:18:26.353 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:26.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7ccab8eb-3be4-4832-b99b-ebf0bef4c9d8 2026-02-23T08:18:26.354 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 7ccab8eb-3be4-4832-b99b-ebf0bef4c9d8 2026-02-23T08:18:26.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7ccab8eb-3be4-4832-b99b-ebf0bef4c9d8' 2026-02-23T08:18:26.355 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:26.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBSDZxp8IyrFRAAw3Kc9OG+aX8V5+/+WjlQbQ== 2026-02-23T08:18:26.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBSDZxp8IyrFRAAw3Kc9OG+aX8V5+/+WjlQbQ=="}' 2026-02-23T08:18:26.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7ccab8eb-3be4-4832-b99b-ebf0bef4c9d8 -i td/test-erasure-eio/0/new.json 2026-02-23T08:18:26.478 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:26.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:18:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBSDZxp8IyrFRAAw3Kc9OG+aX8V5+/+WjlQbQ== --osd-uuid 7ccab8eb-3be4-4832-b99b-ebf0bef4c9d8 2026-02-23T08:18:26.502 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:26.497+0000 7f38a0929a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:26.503 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:26.497+0000 7f38a0929a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:26.504 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:26.497+0000 7f38a0929a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:26.505 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:26.501+0000 7f38a0929a40 -1 bdev(0x556084d36400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:26.505 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:26.501+0000 7f38a0929a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:18:27.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:18:27.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:27.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:18:27.161 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:18:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:27.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:18:27.273 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:18:27.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:27.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:27.274 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:27.274 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:27.289 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:27.285+0000 7f2553f95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:27.290 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:27.285+0000 7f2553f95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:27.291 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:27.285+0000 7f2553f95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:27.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:27.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:18:27.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:27.734 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:27.729+0000 7f2553f95a40 -1 Falling back to public interface 2026-02-23T08:18:28.282 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:28.277+0000 7f2553f95a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:18:28.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:28.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:28.541 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:28.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:28.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:28.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:18:28.759 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:29.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:29.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:29.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:18:29.761 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:29.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:29.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:18:29.977 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4045872780,v1:127.0.0.1:6803/4045872780] [v2:127.0.0.1:6804/4045872780,v1:127.0.0.1:6805/4045872780] exists,up 7ccab8eb-3be4-4832-b99b-ebf0bef4c9d8 2026-02-23T08:18:29.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:29.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:29.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:29.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:29.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:29.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:18:29.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:29.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fe882fa4-1a0a-4298-965e-1bbee69ceaff 2026-02-23T08:18:29.985 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 fe882fa4-1a0a-4298-965e-1bbee69ceaff 2026-02-23T08:18:29.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 fe882fa4-1a0a-4298-965e-1bbee69ceaff' 2026-02-23T08:18:29.986 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:29.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBVDZxpC9ApOxAA9ulm9UhQsiNK6hy5NXHEvQ== 2026-02-23T08:18:29.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBVDZxpC9ApOxAA9ulm9UhQsiNK6hy5NXHEvQ=="}' 2026-02-23T08:18:29.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fe882fa4-1a0a-4298-965e-1bbee69ceaff -i td/test-erasure-eio/1/new.json 2026-02-23T08:18:30.214 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:30.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:18:30.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBVDZxpC9ApOxAA9ulm9UhQsiNK6hy5NXHEvQ== --osd-uuid fe882fa4-1a0a-4298-965e-1bbee69ceaff 2026-02-23T08:18:30.250 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:30.245+0000 7fe0c4a55a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:30.252 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:30.245+0000 7fe0c4a55a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:30.253 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:30.249+0000 7fe0c4a55a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:30.253 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:30.249+0000 7fe0c4a55a40 -1 bdev(0x55d7f9b2a400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:30.253 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:30.249+0000 7fe0c4a55a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:18:30.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:18:30.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:30.885 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:18:30.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:18:30.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:31.188 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:18:31.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:18:31.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:31.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:31.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:31.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:31.211 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:31.205+0000 7f32594a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:31.213 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:31.209+0000 7f32594a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:31.214 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:31.209+0000 7f32594a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:31.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:18:31.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:31.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:18:31.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:31.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:31.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:31.405 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:31.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:31.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:31.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:18:31.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:31.630 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:31.625+0000 7f32594a5a40 -1 Falling back to public interface 2026-02-23T08:18:32.214 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:32.209+0000 7f32594a5a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:18:32.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:32.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:32.627 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:32.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:32.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:32.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:18:32.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:33.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:33.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:33.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:18:33.849 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:33.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:33.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:18:34.063 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/623158403,v1:127.0.0.1:6811/623158403] [v2:127.0.0.1:6812/623158403,v1:127.0.0.1:6813/623158403] exists,up fe882fa4-1a0a-4298-965e-1bbee69ceaff 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:34.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:34.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:34.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:34.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:34.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:18:34.066 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:34.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5de864f6-8fc7-4a91-9fe9-201cde4d018e 2026-02-23T08:18:34.067 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 5de864f6-8fc7-4a91-9fe9-201cde4d018e 2026-02-23T08:18:34.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5de864f6-8fc7-4a91-9fe9-201cde4d018e' 2026-02-23T08:18:34.067 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:34.079 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBaDZxpxjihBBAANUMcFzTIw1St0rujTW/7rg== 2026-02-23T08:18:34.079 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBaDZxpxjihBBAANUMcFzTIw1St0rujTW/7rg=="}' 2026-02-23T08:18:34.079 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5de864f6-8fc7-4a91-9fe9-201cde4d018e -i td/test-erasure-eio/2/new.json 2026-02-23T08:18:34.308 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:34.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:18:34.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBaDZxpxjihBBAANUMcFzTIw1St0rujTW/7rg== --osd-uuid 5de864f6-8fc7-4a91-9fe9-201cde4d018e 2026-02-23T08:18:34.335 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:34.329+0000 7f2780a4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:34.336 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:34.329+0000 7f2780a4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:34.337 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:34.333+0000 7f2780a4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:34.338 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:34.333+0000 7f2780a4fa40 -1 bdev(0x559f74e00400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:34.338 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:34.333+0000 7f2780a4fa40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:18:34.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:18:34.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:34.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:18:34.949 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:18:34.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:35.246 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:18:35.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:18:35.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:35.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:35.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:35.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:35.263 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:35.257+0000 7f34dbc90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:35.264 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:35.257+0000 7f34dbc90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:35.265 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:35.261+0000 7f34dbc90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:35.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:18:35.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:35.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:18:35.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:35.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:35.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:35.468 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:35.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:35.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:35.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:18:35.607 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:35.601+0000 7f34dbc90a40 -1 Falling back to public interface 2026-02-23T08:18:35.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:36.239 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:36.233+0000 7f34dbc90a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:18:36.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:36.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:36.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:36.687 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:36.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:36.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:18:36.907 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3449549521,v1:127.0.0.1:6819/3449549521] [v2:127.0.0.1:6820/3449549521,v1:127.0.0.1:6821/3449549521] exists,up 5de864f6-8fc7-4a91-9fe9-201cde4d018e 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:36.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:36.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:36.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c186addf-9733-4ba4-9333-ac7f90eaf9b4 2026-02-23T08:18:36.913 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 c186addf-9733-4ba4-9333-ac7f90eaf9b4 2026-02-23T08:18:36.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c186addf-9733-4ba4-9333-ac7f90eaf9b4' 2026-02-23T08:18:36.913 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:36.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBcDZxpHAr9NhAAqmN62GTmjyNybRak3Zr+nQ== 2026-02-23T08:18:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBcDZxpHAr9NhAAqmN62GTmjyNybRak3Zr+nQ=="}' 2026-02-23T08:18:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c186addf-9733-4ba4-9333-ac7f90eaf9b4 -i td/test-erasure-eio/3/new.json 2026-02-23T08:18:37.150 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:18:37.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:18:37.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBcDZxpHAr9NhAAqmN62GTmjyNybRak3Zr+nQ== --osd-uuid c186addf-9733-4ba4-9333-ac7f90eaf9b4 2026-02-23T08:18:37.178 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:37.173+0000 7f1e59a67a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:37.179 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:37.173+0000 7f1e59a67a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:37.180 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:37.173+0000 7f1e59a67a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:37.181 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:37.177+0000 7f1e59a67a40 -1 bdev(0x55cd458d6400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:37.181 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:37.177+0000 7f1e59a67a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:18:37.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:18:37.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:37.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:18:37.906 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:18:37.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:38.208 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:18:38.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:18:38.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:38.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:38.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:38.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:38.224 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:38.217+0000 7f8cc58e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:38.226 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:38.221+0000 7f8cc58e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:38.227 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:38.221+0000 7f8cc58e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:38.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:18:38.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:38.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:18:38.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:38.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:38.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:38.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:38.418 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:38.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:38.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:18:38.590 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:38.585+0000 7f8cc58e0a40 -1 Falling back to public interface 2026-02-23T08:18:38.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:39.179 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:39.173+0000 7f8cc58e0a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:18:39.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:39.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:39.634 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:39.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:39.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:39.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:18:39.842 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 24 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2837143875,v1:127.0.0.1:6827/2837143875] [v2:127.0.0.1:6828/2837143875,v1:127.0.0.1:6829/2837143875] exists,up c186addf-9733-4ba4-9333-ac7f90eaf9b4 2026-02-23T08:18:39.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:39.844 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:39.845 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:39.845 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:39.845 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:39.845 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:39.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:39.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-23T08:18:39.846 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:39.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=42d6d5ba-260f-4b1a-92e4-0f780c2a21bd 2026-02-23T08:18:39.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 42d6d5ba-260f-4b1a-92e4-0f780c2a21bd' 2026-02-23T08:18:39.848 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 42d6d5ba-260f-4b1a-92e4-0f780c2a21bd 2026-02-23T08:18:39.848 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:39.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBfDZxpOngdMxAACV731nTmqMx53rM5BI4XLA== 2026-02-23T08:18:39.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBfDZxpOngdMxAACV731nTmqMx53rM5BI4XLA=="}' 2026-02-23T08:18:39.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 42d6d5ba-260f-4b1a-92e4-0f780c2a21bd -i td/test-erasure-eio/4/new.json 2026-02-23T08:18:40.088 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:18:40.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-23T08:18:40.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBfDZxpOngdMxAACV731nTmqMx53rM5BI4XLA== --osd-uuid 42d6d5ba-260f-4b1a-92e4-0f780c2a21bd 2026-02-23T08:18:40.117 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:40.113+0000 7ffbeb420a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:40.119 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:40.113+0000 7ffbeb420a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:40.120 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:40.113+0000 7ffbeb420a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:40.120 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:40.113+0000 7ffbeb420a40 -1 bdev(0x55d0dba5e400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:40.121 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:40.113+0000 7ffbeb420a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-23T08:18:40.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-23T08:18:40.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:40.764 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:18:40.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:18:40.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:41.065 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:18:41.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:18:41.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:41.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:41.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:41.066 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:41.088 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:41.081+0000 7f7fd2e95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:41.090 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:41.085+0000 7f7fd2e95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:41.090 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:41.085+0000 7f7fd2e95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:41.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:18:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:18:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:41.279 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:41.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:41.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:18:41.482 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:41.477+0000 7f7fd2e95a40 -1 Falling back to public interface 2026-02-23T08:18:41.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:42.070 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:42.065+0000 7f7fd2e95a40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:18:42.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:42.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:42.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:42.498 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:42.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:42.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:18:42.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:43.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:43.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:43.718 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:43.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:18:43.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:43.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:18:43.934 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 32 up_thru 33 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3930033450,v1:127.0.0.1:6835/3930033450] [v2:127.0.0.1:6836/3930033450,v1:127.0.0.1:6837/3930033450] exists,up 42d6d5ba-260f-4b1a-92e4-0f780c2a21bd 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:43.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-23T08:18:43.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:43.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:43.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:43.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:43.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-23T08:18:43.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-23T08:18:43.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:43.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:43.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:43.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:43.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:43.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:43.940 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:43.941 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:43.941 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:43.941 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:43.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:43.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:43.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:43.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:43.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:43.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:43.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:43.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:43.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:43.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:43.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:43.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:43.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-23T08:18:43.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:43.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f851db01-fa44-4cfc-90d5-e20345bb42cd 2026-02-23T08:18:43.945 INFO:tasks.workunit.client.0.trial186.stdout:add osd5 f851db01-fa44-4cfc-90d5-e20345bb42cd 2026-02-23T08:18:43.945 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 f851db01-fa44-4cfc-90d5-e20345bb42cd' 2026-02-23T08:18:43.946 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:43.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBjDZxpVcaYOBAACTLAAdjT66XHA1wyWHDyKQ== 2026-02-23T08:18:43.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBjDZxpVcaYOBAACTLAAdjT66XHA1wyWHDyKQ=="}' 2026-02-23T08:18:43.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f851db01-fa44-4cfc-90d5-e20345bb42cd -i td/test-erasure-eio/5/new.json 2026-02-23T08:18:44.173 INFO:tasks.workunit.client.0.trial186.stdout:5 2026-02-23T08:18:44.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-23T08:18:44.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBjDZxpVcaYOBAACTLAAdjT66XHA1wyWHDyKQ== --osd-uuid f851db01-fa44-4cfc-90d5-e20345bb42cd 2026-02-23T08:18:44.203 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:44.197+0000 7f7349e9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:44.204 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:44.197+0000 7f7349e9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:44.206 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:44.201+0000 7f7349e9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:44.207 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:44.201+0000 7f7349e9ea40 -1 bdev(0x559d94d3a400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:44.207 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:44.201+0000 7f7349e9ea40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-23T08:18:44.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-23T08:18:44.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:44.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-23T08:18:44.836 INFO:tasks.workunit.client.0.trial186.stdout:adding osd5 key to auth repository 2026-02-23T08:18:44.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:45.138 INFO:tasks.workunit.client.0.trial186.stdout:start osd.5 2026-02-23T08:18:45.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-23T08:18:45.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:45.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:45.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:45.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:45.162 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:45.156+0000 7f2a87ebba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:45.164 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:45.156+0000 7f2a87ebba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:45.164 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:45.156+0000 7f2a87ebba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:45.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-23T08:18:45.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:45.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-23T08:18:45.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:45.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:45.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:45.352 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:45.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:45.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:45.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:18:45.567 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:45.560+0000 7f2a87ebba40 -1 Falling back to public interface 2026-02-23T08:18:45.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:46.130 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:46.124+0000 7f2a87ebba40 -1 osd.5 0 log_to_monitors true 2026-02-23T08:18:46.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:46.573 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:46.573 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:46.573 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:46.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:46.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:18:46.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:47.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:47.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:47.792 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:18:47.793 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:18:47.793 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:47.793 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:18:48.008 INFO:tasks.workunit.client.0.trial186.stdout:osd.5 up in weight 1 up_from 40 up_thru 41 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/4264738651,v1:127.0.0.1:6843/4264738651] [v2:127.0.0.1:6844/4264738651,v1:127.0.0.1:6845/4264738651] exists,up f851db01-fa44-4cfc-90d5-e20345bb42cd 2026-02-23T08:18:48.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:48.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:48.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:48.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:48.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-23T08:18:48.017 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:48.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=829c4706-ac21-4ad4-adb2-72c67c2806ed 2026-02-23T08:18:48.017 INFO:tasks.workunit.client.0.trial186.stdout:add osd6 829c4706-ac21-4ad4-adb2-72c67c2806ed 2026-02-23T08:18:48.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 829c4706-ac21-4ad4-adb2-72c67c2806ed' 2026-02-23T08:18:48.018 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:48.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBoDZxp/g1MARAA/zT6sQ0dIDaBa1EsZ23rJA== 2026-02-23T08:18:48.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBoDZxp/g1MARAA/zT6sQ0dIDaBa1EsZ23rJA=="}' 2026-02-23T08:18:48.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 829c4706-ac21-4ad4-adb2-72c67c2806ed -i td/test-erasure-eio/6/new.json 2026-02-23T08:18:48.252 INFO:tasks.workunit.client.0.trial186.stdout:6 2026-02-23T08:18:48.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-23T08:18:48.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBoDZxp/g1MARAA/zT6sQ0dIDaBa1EsZ23rJA== --osd-uuid 829c4706-ac21-4ad4-adb2-72c67c2806ed 2026-02-23T08:18:48.279 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:48.272+0000 7f9edae1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:48.280 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:48.272+0000 7f9edae1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:48.281 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:48.276+0000 7f9edae1ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:48.282 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:48.276+0000 7f9edae1ba40 -1 bdev(0x5570c3968400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:48.282 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:48.276+0000 7f9edae1ba40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-23T08:18:48.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-23T08:18:48.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:48.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-23T08:18:48.904 INFO:tasks.workunit.client.0.trial186.stdout:adding osd6 key to auth repository 2026-02-23T08:18:48.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:49.196 INFO:tasks.workunit.client.0.trial186.stdout:start osd.6 2026-02-23T08:18:49.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-23T08:18:49.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:49.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:49.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:49.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:49.212 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:49.204+0000 7f03775a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:49.214 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:49.208+0000 7f03775a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:49.214 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:49.208+0000 7f03775a5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:49.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-23T08:18:49.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:49.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-23T08:18:49.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:49.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:49.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:49.416 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:49.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:49.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:49.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:18:49.563 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:49.556+0000 7f03775a5a40 -1 Falling back to public interface 2026-02-23T08:18:49.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:50.207 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:50.200+0000 7f03775a5a40 -1 osd.6 0 log_to_monitors true 2026-02-23T08:18:50.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:50.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:50.630 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:50.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:50.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:50.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:18:50.851 INFO:tasks.workunit.client.0.trial186.stdout:osd.6 up in weight 1 up_from 47 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1707740136,v1:127.0.0.1:6851/1707740136] [v2:127.0.0.1:6852/1707740136,v1:127.0.0.1:6853/1707740136] exists,up 829c4706-ac21-4ad4-adb2-72c67c2806ed 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:50.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:50.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-23T08:18:50.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:50.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=14898273-d470-4f0e-8c93-c591c054e288 2026-02-23T08:18:50.855 INFO:tasks.workunit.client.0.trial186.stdout:add osd7 14898273-d470-4f0e-8c93-c591c054e288 2026-02-23T08:18:50.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 14898273-d470-4f0e-8c93-c591c054e288' 2026-02-23T08:18:50.856 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:50.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBqDZxp2t2gMxAAupaS4AWiRhUYPUa87TyGxg== 2026-02-23T08:18:50.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBqDZxp2t2gMxAAupaS4AWiRhUYPUa87TyGxg=="}' 2026-02-23T08:18:50.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 14898273-d470-4f0e-8c93-c591c054e288 -i td/test-erasure-eio/7/new.json 2026-02-23T08:18:51.097 INFO:tasks.workunit.client.0.trial186.stdout:7 2026-02-23T08:18:51.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-23T08:18:51.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBqDZxp2t2gMxAAupaS4AWiRhUYPUa87TyGxg== --osd-uuid 14898273-d470-4f0e-8c93-c591c054e288 2026-02-23T08:18:51.124 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:51.116+0000 7f5afe991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:51.126 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:51.120+0000 7f5afe991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:51.127 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:51.120+0000 7f5afe991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:51.127 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:51.120+0000 7f5afe991a40 -1 bdev(0x556cf0112400 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:51.127 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:51.120+0000 7f5afe991a40 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-23T08:18:51.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-23T08:18:51.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:51.785 INFO:tasks.workunit.client.0.trial186.stdout:adding osd7 key to auth repository 2026-02-23T08:18:51.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-23T08:18:51.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:52.087 INFO:tasks.workunit.client.0.trial186.stdout:start osd.7 2026-02-23T08:18:52.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-23T08:18:52.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:52.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:52.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:52.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:52.112 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:52.104+0000 7f4725811a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:52.113 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:52.108+0000 7f4725811a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:52.114 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:52.108+0000 7f4725811a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:52.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-23T08:18:52.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:52.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-23T08:18:52.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:52.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:52.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:52.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:52.299 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:52.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:52.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-23T08:18:52.475 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:52.468+0000 7f4725811a40 -1 Falling back to public interface 2026-02-23T08:18:52.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:53.066 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:53.060+0000 7f4725811a40 -1 osd.7 0 log_to_monitors true 2026-02-23T08:18:53.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:53.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:53.520 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:53.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:53.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:53.521 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-23T08:18:53.744 INFO:tasks.workunit.client.0.trial186.stdout:osd.7 up in weight 1 up_from 55 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/1938392353,v1:127.0.0.1:6859/1938392353] [v2:127.0.0.1:6860/1938392353,v1:127.0.0.1:6861/1938392353] exists,up 14898273-d470-4f0e-8c93-c591c054e288 2026-02-23T08:18:53.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:18:53.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:53.749 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:18:53.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:18:53.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:18:53.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:18:53.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:18:53.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:18:53.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:18:53.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-23T08:18:53.751 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:18:53.752 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bbe5aede-9369-4878-941c-3fad6a6b0a41 2026-02-23T08:18:53.752 INFO:tasks.workunit.client.0.trial186.stdout:add osd8 bbe5aede-9369-4878-941c-3fad6a6b0a41 2026-02-23T08:18:53.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 bbe5aede-9369-4878-941c-3fad6a6b0a41' 2026-02-23T08:18:53.753 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:18:53.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBtDZxpbQ4wLRAAtYuOmAMJL8jf9Ba5wT7IQQ== 2026-02-23T08:18:53.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBtDZxpbQ4wLRAAtYuOmAMJL8jf9Ba5wT7IQQ=="}' 2026-02-23T08:18:53.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bbe5aede-9369-4878-941c-3fad6a6b0a41 -i td/test-erasure-eio/8/new.json 2026-02-23T08:18:53.988 INFO:tasks.workunit.client.0.trial186.stdout:8 2026-02-23T08:18:53.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-23T08:18:53.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBtDZxpbQ4wLRAAtYuOmAMJL8jf9Ba5wT7IQQ== --osd-uuid bbe5aede-9369-4878-941c-3fad6a6b0a41 2026-02-23T08:18:54.016 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.008+0000 7f0398dcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:54.017 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.008+0000 7f0398dcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:54.018 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.012+0000 7f0398dcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:54.018 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.012+0000 7f0398dcfa40 -1 bdev(0x56184c3b8400 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-23T08:18:54.018 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.012+0000 7f0398dcfa40 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-23T08:18:54.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-23T08:18:54.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:18:54.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-23T08:18:54.674 INFO:tasks.workunit.client.0.trial186.stdout:adding osd8 key to auth repository 2026-02-23T08:18:54.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:18:54.966 INFO:tasks.workunit.client.0.trial186.stdout:start osd.8 2026-02-23T08:18:54.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-23T08:18:54.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:18:54.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:18:54.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:18:54.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:18:54.990 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.984+0000 7fc39cecba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:54.991 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.984+0000 7fc39cecba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:54.992 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:54.984+0000 7fc39cecba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:18:55.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-23T08:18:55.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:18:55.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-23T08:18:55.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:18:55.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:18:55.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:55.191 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:18:55.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:18:55.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:55.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-23T08:18:55.351 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:55.344+0000 7fc39cecba40 -1 Falling back to public interface 2026-02-23T08:18:55.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:18:55.983 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:18:55.976+0000 7fc39cecba40 -1 osd.8 0 log_to_monitors true 2026-02-23T08:18:56.411 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:18:56.411 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:18:56.411 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:18:56.412 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:18:56.412 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:18:56.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-23T08:18:56.628 INFO:tasks.workunit.client.0.trial186.stdout:osd.8 up in weight 1 up_from 64 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/2250195878,v1:127.0.0.1:6867/2250195878] [v2:127.0.0.1:6868/2250195878,v1:127.0.0.1:6869/2250195878] exists,up bbe5aede-9369-4878-941c-3fad6a6b0a41 2026-02-23T08:18:56.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:18:56.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:18:56.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:18:56.629 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:18:56.629 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:18:56.629 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:18:56.629 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:18:56.629 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:18:56.629 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:18:56.630 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:18:56.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:18:56.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:18:56.694 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:18:56.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:18:56.703 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:18:27.733+0000 7f2553f95a40 0 load: jerasure load: lrc 2026-02-23T08:18:56.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-23T08:18:56.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-23T08:18:56.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:18:56.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:18:56.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-23T08:18:56.705 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:18:56.705 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-23T08:18:56.705 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:18:56.705 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-23T08:18:56.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:18:56.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:18:57.381 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:18:57.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:18:58.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:18:58.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:18:58.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:18:58.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:18:58.394 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:18:58.395 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:18:58.395 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:18:58.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:18:58.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:18:58.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:18:58.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:18:58.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:18:58.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:18:58.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:18:58.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:18:58.494 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:18:58.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:18:58.710 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:8' 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:58.711 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:18:58.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-23T08:18:58.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-23T08:18:58.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-23T08:18:58.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:58.791 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:18:58.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607560 2026-02-23T08:18:58.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607560 2026-02-23T08:18:58.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560' 2026-02-23T08:18:58.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:58.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:18:58.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444039 2026-02-23T08:18:58.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444039 2026-02-23T08:18:58.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039' 2026-02-23T08:18:58.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:58.951 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:18:59.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215110 2026-02-23T08:18:59.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215110 2026-02-23T08:18:59.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039 3-103079215110' 2026-02-23T08:18:59.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:59.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:18:59.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953478 2026-02-23T08:18:59.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953478 2026-02-23T08:18:59.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039 3-103079215110 4-137438953478' 2026-02-23T08:18:59.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:59.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:18:59.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691845 2026-02-23T08:18:59.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691845 2026-02-23T08:18:59.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039 3-103079215110 4-137438953478 5-171798691845' 2026-02-23T08:18:59.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:59.192 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:18:59.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462916 2026-02-23T08:18:59.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462916 2026-02-23T08:18:59.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039 3-103079215110 4-137438953478 5-171798691845 6-201863462916' 2026-02-23T08:18:59.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:59.272 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:18:59.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=236223201284 2026-02-23T08:18:59.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 236223201284 2026-02-23T08:18:59.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039 3-103079215110 4-137438953478 5-171798691845 6-201863462916 7-236223201284' 2026-02-23T08:18:59.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:18:59.349 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:18:59.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=274877906947 2026-02-23T08:18:59.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 274877906947 2026-02-23T08:18:59.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039 3-103079215110 4-137438953478 5-171798691845 6-201863462916 7-236223201284 8-274877906947' 2026-02-23T08:18:59.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:59.432 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-23T08:18:59.432 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:59.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:18:59.433 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-23T08:18:59.433 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:59.435 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-23T08:18:59.435 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803784 2026-02-23T08:18:59.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-23T08:18:59.436 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:18:59.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803784 2026-02-23T08:18:59.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:59.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607560 2026-02-23T08:18:59.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:59.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:18:59.661 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607560 2026-02-23T08:18:59.661 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:59.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607560 2026-02-23T08:18:59.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607560' 2026-02-23T08:18:59.663 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607560 2026-02-23T08:18:59.663 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:18:59.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607560 -lt 51539607560 2026-02-23T08:18:59.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:18:59.893 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444039 2026-02-23T08:18:59.893 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:18:59.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:18:59.895 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444039 2026-02-23T08:18:59.895 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:18:59.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444039 2026-02-23T08:18:59.897 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444039 2026-02-23T08:18:59.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444039' 2026-02-23T08:18:59.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:19:00.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-23T08:19:00.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:00.117 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-103079215110 2026-02-23T08:19:00.117 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:00.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:19:00.118 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-103079215110 2026-02-23T08:19:00.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:00.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=103079215110 2026-02-23T08:19:00.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 103079215110' 2026-02-23T08:19:00.121 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 103079215110 2026-02-23T08:19:00.121 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:19:00.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 103079215110 -lt 103079215110 2026-02-23T08:19:00.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:00.349 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-137438953478 2026-02-23T08:19:00.349 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:00.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:19:00.350 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-137438953478 2026-02-23T08:19:00.350 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:00.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=137438953478 2026-02-23T08:19:00.352 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 137438953478 2026-02-23T08:19:00.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 137438953478' 2026-02-23T08:19:00.353 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:19:00.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 137438953478 -lt 137438953478 2026-02-23T08:19:00.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:00.584 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-171798691845 2026-02-23T08:19:00.584 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:00.585 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:19:00.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-171798691845 2026-02-23T08:19:00.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:00.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=171798691845 2026-02-23T08:19:00.587 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 171798691845 2026-02-23T08:19:00.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 171798691845' 2026-02-23T08:19:00.587 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:19:00.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 171798691845 -lt 171798691845 2026-02-23T08:19:00.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:00.799 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-201863462916 2026-02-23T08:19:00.799 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:00.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:19:00.801 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-201863462916 2026-02-23T08:19:00.801 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:00.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462916 2026-02-23T08:19:00.803 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 201863462916 2026-02-23T08:19:00.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 201863462916' 2026-02-23T08:19:00.803 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:19:01.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462916 -lt 201863462916 2026-02-23T08:19:01.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:01.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-236223201284 2026-02-23T08:19:01.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:01.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-23T08:19:01.033 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-236223201284 2026-02-23T08:19:01.033 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:01.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=236223201284 2026-02-23T08:19:01.034 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.7 seq 236223201284 2026-02-23T08:19:01.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 236223201284' 2026-02-23T08:19:01.035 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-23T08:19:01.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 236223201284 -lt 236223201284 2026-02-23T08:19:01.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:01.254 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-274877906947 2026-02-23T08:19:01.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:01.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:19:01.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-274877906947 2026-02-23T08:19:01.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:01.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=274877906947 2026-02-23T08:19:01.258 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.8 seq 274877906947 2026-02-23T08:19:01.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 274877906947' 2026-02-23T08:19:01.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:19:01.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 274877906947 -lt 274877906947 2026-02-23T08:19:01.481 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:19:01.481 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:01.481 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:19:01.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:19:01.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:19:01.986 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:19:01.987 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:01.987 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:19:02.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-23T08:19:02.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:19:02.299 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:19:02.516 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr:7' 2026-02-23T08:19:02.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-23T08:19:02.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-23T08:19:02.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:19:02.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-23T08:19:02.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:19:02.518 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:19:02.520 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-23T08:19:02.521 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:19:02.521 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:19:02.521 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:02.521 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:02.521 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:02.521 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:19:02.521 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:19:02.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:19:02.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:19:02.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:19:02.583 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-23T08:19:02.583 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:19:02.584 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:19:02.584 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:02.584 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:02.584 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:02.584 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:19:02.585 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:19:02.585 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:19:02.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:19:02.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=3 2026-02-23T08:19:02.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:19:02.648 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-23T08:19:02.648 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:19:02.648 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:19:02.648 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:19:02.648 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:19:02.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:19:02.869 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:19:02.869 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:19:02.869 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:19:02.869 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:19:02.869 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:19:02.869 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr:7' 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=0 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/0/type 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:19:02.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-23T08:19:02.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:19:02.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-23T08:19:02.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:19:02.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:19:02.871 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:19:02.872 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-23T08:19:02.872 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:19:02.872 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:19:02.872 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:02.872 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:02.872 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:02.872 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:19:02.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:19:02.936 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:19:02.936 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:19:02.937 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:19:02.937 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.0 2026-02-23T08:19:02.937 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:19:02.937 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:19:02.937 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:02.938 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:02.938 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:02.938 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:19:02.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:19:02.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-23T08:19:03.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-23T08:19:03.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:19:03.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:19:03.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=4 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:19:03.001 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:19:03.221 INFO:tasks.workunit.client.0.trial186.stderr:7' 2026-02-23T08:19:03.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-23T08:19:03.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-23T08:19:03.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:19:03.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=6 2026-02-23T08:19:03.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:19:03.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/6/type 2026-02-23T08:19:03.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:19:03.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-23T08:19:03.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.6.asok 2026-02-23T08:19:03.225 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:19:03.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:19:03.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:19:03.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:19:03.288 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.6 2026-02-23T08:19:03.288 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-23T08:19:03.288 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-23T08:19:03.288 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:03.288 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:03.289 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:03.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.6.asok 2026-02-23T08:19:03.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:19:03.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-23T08:19:03.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-23T08:19:03.351 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:19:03.352 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:19:03.352 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:19:03.352 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:19:03.576 INFO:tasks.workunit.client.0.trial186.stderr:7' 2026-02-23T08:19:03.577 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-23T08:19:03.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-23T08:19:03.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-23T08:19:03.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-23T08:19:03.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-23T08:19:03.578 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:19:03.578 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:19:03.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:19:03.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:19:03.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:19:03.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:19:03.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-23T08:19:04.090 INFO:tasks.workunit.client.0.trial186.stderr:osd.7 is already down. 2026-02-23T08:19:04.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-23T08:19:04.347 INFO:tasks.workunit.client.0.trial186.stderr:osd.7 is already out. 2026-02-23T08:19:04.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-23T08:19:04.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:19:04.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:19:04.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:19:04.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:19:04.359 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:19:04.359 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:19:04.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:19:04.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:19:04.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:19:04.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:19:04.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:19:04.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:19:04.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:19:04.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:19:04.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:19:04.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:19:04.696 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:19:04.696 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:19:04.696 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:19:04.696 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:19:04.696 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:19:04.696 INFO:tasks.workunit.client.0.trial186.stderr:6 2026-02-23T08:19:04.697 INFO:tasks.workunit.client.0.trial186.stderr:7 2026-02-23T08:19:04.697 INFO:tasks.workunit.client.0.trial186.stderr:8' 2026-02-23T08:19:04.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:19:04.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:04.697 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:19:04.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-23T08:19:04.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-23T08:19:04.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788' 2026-02-23T08:19:04.776 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:04.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:19:04.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607563 2026-02-23T08:19:04.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607563 2026-02-23T08:19:04.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-51539607563' 2026-02-23T08:19:04.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:04.856 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:19:04.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444042 2026-02-23T08:19:04.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444042 2026-02-23T08:19:04.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-73014444042' 2026-02-23T08:19:04.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:04.934 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:19:05.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215113 2026-02-23T08:19:05.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215113 2026-02-23T08:19:05.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-73014444042 3-103079215113' 2026-02-23T08:19:05.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:05.016 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:19:05.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953481 2026-02-23T08:19:05.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953481 2026-02-23T08:19:05.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-73014444042 3-103079215113 4-137438953481' 2026-02-23T08:19:05.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:05.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:19:05.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691848 2026-02-23T08:19:05.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691848 2026-02-23T08:19:05.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-73014444042 3-103079215113 4-137438953481 5-171798691848' 2026-02-23T08:19:05.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:05.178 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:19:05.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462919 2026-02-23T08:19:05.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462919 2026-02-23T08:19:05.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-73014444042 3-103079215113 4-137438953481 5-171798691848 6-201863462919' 2026-02-23T08:19:05.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:05.260 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-23T08:19:05.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-23T08:19:05.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-23T08:19:05.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-23T08:19:05.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-23T08:19:05.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:05.319 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-23T08:19:05.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=274877906950 2026-02-23T08:19:05.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 274877906950 2026-02-23T08:19:05.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-51539607563 2-73014444042 3-103079215113 4-137438953481 5-171798691848 6-201863462919 8-274877906950' 2026-02-23T08:19:05.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:05.396 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803788 2026-02-23T08:19:05.396 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:05.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:19:05.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-23T08:19:05.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:05.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803788 2026-02-23T08:19:05.399 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803788 2026-02-23T08:19:05.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-23T08:19:05.400 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:19:05.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-23T08:19:05.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:05.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607563 2026-02-23T08:19:05.629 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:05.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:19:05.630 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607563 2026-02-23T08:19:05.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:05.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607563 2026-02-23T08:19:05.632 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607563 2026-02-23T08:19:05.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607563' 2026-02-23T08:19:05.632 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:19:05.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607563 -lt 51539607563 2026-02-23T08:19:05.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:05.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444042 2026-02-23T08:19:05.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:05.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:19:05.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444042 2026-02-23T08:19:05.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:05.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444042 2026-02-23T08:19:05.856 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444042 2026-02-23T08:19:05.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444042' 2026-02-23T08:19:05.856 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:19:06.079 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444042 -lt 73014444042 2026-02-23T08:19:06.079 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:06.080 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-103079215113 2026-02-23T08:19:06.080 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:06.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:19:06.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-103079215113 2026-02-23T08:19:06.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:06.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=103079215113 2026-02-23T08:19:06.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 103079215113' 2026-02-23T08:19:06.084 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 103079215113 2026-02-23T08:19:06.084 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:19:06.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 103079215114 -lt 103079215113 2026-02-23T08:19:06.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:06.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-137438953481 2026-02-23T08:19:06.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:06.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:19:06.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-137438953481 2026-02-23T08:19:06.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:06.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=137438953481 2026-02-23T08:19:06.319 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 137438953481 2026-02-23T08:19:06.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 137438953481' 2026-02-23T08:19:06.319 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:19:06.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 137438953481 -lt 137438953481 2026-02-23T08:19:06.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:06.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-171798691848 2026-02-23T08:19:06.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:06.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:19:06.544 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-171798691848 2026-02-23T08:19:06.544 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:06.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=171798691848 2026-02-23T08:19:06.545 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 171798691848 2026-02-23T08:19:06.546 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 171798691848' 2026-02-23T08:19:06.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:19:06.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 171798691848 -lt 171798691848 2026-02-23T08:19:06.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:06.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-201863462919 2026-02-23T08:19:06.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:06.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:19:06.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-201863462919 2026-02-23T08:19:06.776 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:06.777 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462919 2026-02-23T08:19:06.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 201863462919' 2026-02-23T08:19:06.778 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 201863462919 2026-02-23T08:19:06.778 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:19:06.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-23T08:19:06.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:06.992 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-274877906950 2026-02-23T08:19:06.993 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:06.993 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-23T08:19:06.994 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-274877906950 2026-02-23T08:19:06.994 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:06.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=274877906950 2026-02-23T08:19:06.996 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.8 seq 274877906950 2026-02-23T08:19:06.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 274877906950' 2026-02-23T08:19:06.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-23T08:19:07.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 274877906950 -lt 274877906950 2026-02-23T08:19:07.216 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:19:07.216 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:07.216 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:07.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:19:07.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:19:07.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:19:07.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:19:07.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:19:07.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:19:07.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:19:07.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:19:07.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:19:07.726 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:19:07.726 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:07.726 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:08.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:19:08.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:19:08.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:19:08.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:19:08.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:19:08.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:19:08.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-23T08:19:08.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:19:08.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:19:08.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-23T08:19:08.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:19:08.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:19:08.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:19:08.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:19:08.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:19:08.301 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:19:08.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:19:08.593 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:19:08.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:19:08.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:19:08.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:19:08.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:19:08.605 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:19:08.605 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:19:08.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:19:08.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:19:08.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:19:08.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:19:08.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:19:08.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:19:08.720 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:19:08.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:19:08.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:19:08.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:19:08.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:08.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:19:08.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:19:08.724 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:08.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:19:08.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:19:08.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:19:08.753 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:19:08.753 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:08.753 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:08.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:19:08.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:19:08.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:19:08.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:19:08.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:19:08.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:19:08.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:19:08.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:19:08.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:19:08.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:19:08.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:19:08.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:19:08.760 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:19:08.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:08.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:19:08.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:19:08.762 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:08.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:19:08.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:19:08.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:19:08.766 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:19:08.766 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:08.766 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:08.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:19:08.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:19:08.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:19:08.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:19:08.769 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:19:08.769 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:08.769 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:08.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:19:08.771 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:19:08.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:19:08.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:19:08.794 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:19:08.794 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:08.794 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:08.794 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:08.794 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:08.794 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:08.794 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:08.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:19:08.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:19:08.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:19:08.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:19:08.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:19:08.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:19:08.826 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:19:08.827 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:19:08.827 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:19:08.827 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:19:08.827 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:08.827 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:08.827 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:08.827 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:19:08.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:19:08.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:19:08.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:19:08.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:19:08.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:19:08.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:19:08.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:19:08.884 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:19:08.884 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:19:08.884 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:19:08.885 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:08.886 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:08.886 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:08.886 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:19:08.886 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:19:08.886 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:19:08.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:19:08.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:19:08.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:19:08.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:19:08.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:19:08.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:19:08.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:19:09.051 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:19:09.051 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:09.051 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:09.052 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:09.052 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:09.052 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:09.052 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:09.053 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:19:09.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:19:09.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:19:09.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:19:09.190 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:19:09.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:19:10.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:19:10.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:19:10.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:19:10.200 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:10.200 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:10.200 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:10.201 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:19:10.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:19:10.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:19:10.249 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:19:10.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:19:10.257 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:19:08.808+0000 7f22fc787ec0 0 load: jerasure load: lrc 2026-02-23T08:19:10.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:19:10.259 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-23T08:19:10.260 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-23T08:19:10.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:10.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:19:10.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:10.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:10.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:10.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:10.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:10.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:10.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:10.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:10.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:10.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:10.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:10.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:19:10.267 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:10.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d6908d37-9fa4-4d30-9b91-e2101d706124 2026-02-23T08:19:10.267 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 d6908d37-9fa4-4d30-9b91-e2101d706124 2026-02-23T08:19:10.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d6908d37-9fa4-4d30-9b91-e2101d706124' 2026-02-23T08:19:10.268 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:10.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB+DZxprlZaEBAAvXGv0RyBkZnvfMHZH/38fA== 2026-02-23T08:19:10.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB+DZxprlZaEBAAvXGv0RyBkZnvfMHZH/38fA=="}' 2026-02-23T08:19:10.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d6908d37-9fa4-4d30-9b91-e2101d706124 -i td/test-erasure-eio/0/new.json 2026-02-23T08:19:10.392 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:10.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:19:10.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB+DZxprlZaEBAAvXGv0RyBkZnvfMHZH/38fA== --osd-uuid d6908d37-9fa4-4d30-9b91-e2101d706124 2026-02-23T08:19:10.417 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:10.408+0000 7f1e1739aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:10.419 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:10.412+0000 7f1e1739aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:10.420 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:10.412+0000 7f1e1739aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:10.420 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:10.412+0000 7f1e1739aa40 -1 bdev(0x55a137618400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:10.421 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:10.412+0000 7f1e1739aa40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:19:11.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:19:11.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:11.126 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:19:11.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:19:11.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:11.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:19:11.248 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:19:11.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:11.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:11.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:11.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:11.307 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:11.300+0000 7f30a1dc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:11.309 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:11.300+0000 7f30a1dc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:11.310 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:11.304+0000 7f30a1dc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:11.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:19:11.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:11.683 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:11.676+0000 7f30a1dc1a40 -1 Falling back to public interface 2026-02-23T08:19:12.230 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:12.224+0000 7f30a1dc1a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:19:12.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:12.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:12.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:12.518 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:12.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:12.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:19:12.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:13.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:13.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:13.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:19:13.733 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:13.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:13.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:19:13.957 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/977263096,v1:127.0.0.1:6803/977263096] [v2:127.0.0.1:6804/977263096,v1:127.0.0.1:6805/977263096] exists,up d6908d37-9fa4-4d30-9b91-e2101d706124 2026-02-23T08:19:13.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:13.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:13.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:13.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:13.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:19:13.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:13.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:13.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:13.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:19:13.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:13.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ece8dc29-53a0-43b3-99ce-3dea92c0e79c 2026-02-23T08:19:13.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 ece8dc29-53a0-43b3-99ce-3dea92c0e79c' 2026-02-23T08:19:13.965 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 ece8dc29-53a0-43b3-99ce-3dea92c0e79c 2026-02-23T08:19:13.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:13.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCBDZxpcYcBOhAAjVRnSY0A+DYG2Y1Cj9ZMSw== 2026-02-23T08:19:13.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCBDZxpcYcBOhAAjVRnSY0A+DYG2Y1Cj9ZMSw=="}' 2026-02-23T08:19:13.976 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ece8dc29-53a0-43b3-99ce-3dea92c0e79c -i td/test-erasure-eio/1/new.json 2026-02-23T08:19:14.205 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:14.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:19:14.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCBDZxpcYcBOhAAjVRnSY0A+DYG2Y1Cj9ZMSw== --osd-uuid ece8dc29-53a0-43b3-99ce-3dea92c0e79c 2026-02-23T08:19:14.243 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:14.236+0000 7fc54374ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:14.244 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:14.236+0000 7fc54374ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:14.245 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:14.240+0000 7fc54374ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:14.246 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:14.240+0000 7fc54374ba40 -1 bdev(0x559f36938400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:14.246 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:14.240+0000 7fc54374ba40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:19:14.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:19:14.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:14.906 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:19:14.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:19:14.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:15.207 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:19:15.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:19:15.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:15.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:15.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:15.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:15.233 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:15.224+0000 7f254a61ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:15.234 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:15.228+0000 7f254a61ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:15.235 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:15.228+0000 7f254a61ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:15.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:19:15.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:15.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:19:15.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:15.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:15.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:15.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:15.426 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:15.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:15.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:19:15.583 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:15.576+0000 7f254a61ea40 -1 Falling back to public interface 2026-02-23T08:19:15.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:16.207 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:16.200+0000 7f254a61ea40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:19:16.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:16.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:16.646 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:16.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:16.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:16.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:19:16.859 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2255323309,v1:127.0.0.1:6811/2255323309] [v2:127.0.0.1:6812/2255323309,v1:127.0.0.1:6813/2255323309] exists,up ece8dc29-53a0-43b3-99ce-3dea92c0e79c 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:19:16.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:16.864 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:16.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:19:16.866 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:16.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=941ecbe1-03f8-4a29-a154-2e69e9307ef3 2026-02-23T08:19:16.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 941ecbe1-03f8-4a29-a154-2e69e9307ef3' 2026-02-23T08:19:16.867 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 941ecbe1-03f8-4a29-a154-2e69e9307ef3 2026-02-23T08:19:16.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:16.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEDZxpiDwbNBAA+hE6DOpokKVTZRokAbA2Aw== 2026-02-23T08:19:16.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEDZxpiDwbNBAA+hE6DOpokKVTZRokAbA2Aw=="}' 2026-02-23T08:19:16.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 941ecbe1-03f8-4a29-a154-2e69e9307ef3 -i td/test-erasure-eio/2/new.json 2026-02-23T08:19:17.110 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:17.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:19:17.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEDZxpiDwbNBAA+hE6DOpokKVTZRokAbA2Aw== --osd-uuid 941ecbe1-03f8-4a29-a154-2e69e9307ef3 2026-02-23T08:19:17.140 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:17.132+0000 7fb24c08fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:17.141 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:17.132+0000 7fb24c08fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:17.142 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:17.136+0000 7fb24c08fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:17.142 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:17.136+0000 7fb24c08fa40 -1 bdev(0x55609eef8400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:17.142 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:17.136+0000 7fb24c08fa40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:19:17.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:19:17.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:17.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:19:17.758 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:19:17.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:18.054 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:19:18.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:19:18.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:18.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:18.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:18.056 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:18.080 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:18.072+0000 7f899a15ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:18.081 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:18.076+0000 7f899a15ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:18.083 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:18.076+0000 7f899a15ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:18.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:19:18.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:18.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:19:18.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:18.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:18.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:18.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:18.276 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:18.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:18.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:19:18.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:18.539 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:18.532+0000 7f899a15ea40 -1 Falling back to public interface 2026-02-23T08:19:19.116 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:19.108+0000 7f899a15ea40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:19:19.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:19.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:19.497 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:19.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:19.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:19.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:19:19.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:20.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:20.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:20.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:19:20.714 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:20.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:20.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:19:20.935 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/163891063,v1:127.0.0.1:6819/163891063] [v2:127.0.0.1:6820/163891063,v1:127.0.0.1:6821/163891063] exists,up 941ecbe1-03f8-4a29-a154-2e69e9307ef3 2026-02-23T08:19:20.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:20.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:20.940 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:20.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:20.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:20.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:20.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:19:20.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:20.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=86c3dd9d-73a3-4e9c-af46-0273e3565e16 2026-02-23T08:19:20.943 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 86c3dd9d-73a3-4e9c-af46-0273e3565e16 2026-02-23T08:19:20.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 86c3dd9d-73a3-4e9c-af46-0273e3565e16' 2026-02-23T08:19:20.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:20.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCIDZxp2jKQOBAAtfwuMbt67aYEDvyuOcMMtw== 2026-02-23T08:19:20.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCIDZxp2jKQOBAAtfwuMbt67aYEDvyuOcMMtw=="}' 2026-02-23T08:19:20.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 86c3dd9d-73a3-4e9c-af46-0273e3565e16 -i td/test-erasure-eio/3/new.json 2026-02-23T08:19:21.182 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:19:21.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:19:21.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCIDZxp2jKQOBAAtfwuMbt67aYEDvyuOcMMtw== --osd-uuid 86c3dd9d-73a3-4e9c-af46-0273e3565e16 2026-02-23T08:19:21.211 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:21.204+0000 7f8dd705fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:21.212 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:21.204+0000 7f8dd705fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:21.214 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:21.208+0000 7f8dd705fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:21.215 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:21.208+0000 7f8dd705fa40 -1 bdev(0x5637c6fba400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:21.215 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:21.208+0000 7f8dd705fa40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:19:21.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:19:21.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:21.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:19:21.858 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:19:21.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:22.155 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:19:22.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:19:22.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:22.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:22.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:22.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:22.181 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:22.172+0000 7f6037c90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:22.183 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:22.176+0000 7f6037c90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:22.184 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:22.176+0000 7f6037c90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:22.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:19:22.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:22.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:19:22.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:22.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:22.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:22.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:22.371 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:22.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:22.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:19:22.571 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:22.564+0000 7f6037c90a40 -1 Falling back to public interface 2026-02-23T08:19:22.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:23.174 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:23.168+0000 7f6037c90a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:19:23.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:23.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:23.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:23.596 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:23.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:23.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:19:23.835 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3075294917,v1:127.0.0.1:6827/3075294917] [v2:127.0.0.1:6828/3075294917,v1:127.0.0.1:6829/3075294917] exists,up 86c3dd9d-73a3-4e9c-af46-0273e3565e16 2026-02-23T08:19:23.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:23.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:23.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:23.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:23.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-23T08:19:23.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-23T08:19:23.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:23.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-23T08:19:23.841 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:23.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c574caa5-b124-4aef-8811-1c84185ea192 2026-02-23T08:19:23.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 c574caa5-b124-4aef-8811-1c84185ea192' 2026-02-23T08:19:23.842 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 c574caa5-b124-4aef-8811-1c84185ea192 2026-02-23T08:19:23.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:23.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCLDZxpJPOrMhAA80KeJI3pUpmopHababBaow== 2026-02-23T08:19:23.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCLDZxpJPOrMhAA80KeJI3pUpmopHababBaow=="}' 2026-02-23T08:19:23.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c574caa5-b124-4aef-8811-1c84185ea192 -i td/test-erasure-eio/4/new.json 2026-02-23T08:19:24.073 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:19:24.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-23T08:19:24.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCLDZxpJPOrMhAA80KeJI3pUpmopHababBaow== --osd-uuid c574caa5-b124-4aef-8811-1c84185ea192 2026-02-23T08:19:24.102 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:24.096+0000 7fa797caaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:24.104 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:24.096+0000 7fa797caaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:24.105 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:24.096+0000 7fa797caaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:24.105 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:24.096+0000 7fa797caaa40 -1 bdev(0x5582bad38400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:24.105 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:24.100+0000 7fa797caaa40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-23T08:19:24.752 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-23T08:19:24.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:24.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:19:24.754 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:19:24.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:25.052 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:19:25.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:19:25.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:25.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:25.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:25.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:25.069 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:25.060+0000 7f45eadf3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:25.071 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:25.064+0000 7f45eadf3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:25.072 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:25.064+0000 7f45eadf3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:19:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:19:25.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:25.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:25.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:25.266 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:25.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:25.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:25.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:19:25.467 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:25.460+0000 7f45eadf3a40 -1 Falling back to public interface 2026-02-23T08:19:25.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:26.087 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:26.080+0000 7f45eadf3a40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:19:26.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:26.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:26.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:26.486 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:26.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:19:26.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:27.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:27.700 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:27.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:27.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:19:27.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:27.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:19:27.920 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 32 up_thru 33 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1404712691,v1:127.0.0.1:6835/1404712691] [v2:127.0.0.1:6836/1404712691,v1:127.0.0.1:6837/1404712691] exists,up c574caa5-b124-4aef-8811-1c84185ea192 2026-02-23T08:19:27.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:27.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:27.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:27.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:27.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:27.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:27.926 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:27.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:27.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:27.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:27.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:27.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-23T08:19:27.928 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:27.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=db7104cd-a90b-49eb-8fb8-a7269eaf6e3e 2026-02-23T08:19:27.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 db7104cd-a90b-49eb-8fb8-a7269eaf6e3e' 2026-02-23T08:19:27.929 INFO:tasks.workunit.client.0.trial186.stdout:add osd5 db7104cd-a90b-49eb-8fb8-a7269eaf6e3e 2026-02-23T08:19:27.930 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:27.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCPDZxp5PHSNxAAUfkjq/XHRjW2U2NnhlIajQ== 2026-02-23T08:19:27.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCPDZxp5PHSNxAAUfkjq/XHRjW2U2NnhlIajQ=="}' 2026-02-23T08:19:27.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new db7104cd-a90b-49eb-8fb8-a7269eaf6e3e -i td/test-erasure-eio/5/new.json 2026-02-23T08:19:28.160 INFO:tasks.workunit.client.0.trial186.stdout:5 2026-02-23T08:19:28.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-23T08:19:28.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCPDZxp5PHSNxAAUfkjq/XHRjW2U2NnhlIajQ== --osd-uuid db7104cd-a90b-49eb-8fb8-a7269eaf6e3e 2026-02-23T08:19:28.198 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:28.188+0000 7f59960caa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:28.199 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:28.192+0000 7f59960caa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:28.200 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:28.192+0000 7f59960caa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:28.201 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:28.192+0000 7f59960caa40 -1 bdev(0x559fe9a5e400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:28.201 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:28.192+0000 7f59960caa40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-23T08:19:28.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-23T08:19:28.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:28.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-23T08:19:28.811 INFO:tasks.workunit.client.0.trial186.stdout:adding osd5 key to auth repository 2026-02-23T08:19:28.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:29.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-23T08:19:29.114 INFO:tasks.workunit.client.0.trial186.stdout:start osd.5 2026-02-23T08:19:29.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:29.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:29.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:29.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:29.131 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:29.124+0000 7fe3e4e9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:29.132 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:29.124+0000 7fe3e4e9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:29.133 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:29.124+0000 7fe3e4e9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:29.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-23T08:19:29.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:29.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-23T08:19:29.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:29.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:29.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:29.323 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:29.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:29.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:29.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:19:29.546 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:29.603 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:29.596+0000 7fe3e4e9da40 -1 Falling back to public interface 2026-02-23T08:19:30.181 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:30.172+0000 7fe3e4e9da40 -1 osd.5 0 log_to_monitors true 2026-02-23T08:19:30.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:30.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:30.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:30.548 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:30.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:30.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:19:30.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:31.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:31.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:31.802 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:31.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:19:31.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:31.803 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:19:32.020 INFO:tasks.workunit.client.0.trial186.stdout:osd.5 up in weight 1 up_from 41 up_thru 42 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2596890116,v1:127.0.0.1:6843/2596890116] [v2:127.0.0.1:6844/2596890116,v1:127.0.0.1:6845/2596890116] exists,up db7104cd-a90b-49eb-8fb8-a7269eaf6e3e 2026-02-23T08:19:32.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:32.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:32.024 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:32.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:32.026 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:32.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:32.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:32.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:32.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:32.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:32.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-23T08:19:32.028 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:32.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0ef4ba12-6f58-4c32-b678-aa25454287c6 2026-02-23T08:19:32.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 0ef4ba12-6f58-4c32-b678-aa25454287c6' 2026-02-23T08:19:32.029 INFO:tasks.workunit.client.0.trial186.stdout:add osd6 0ef4ba12-6f58-4c32-b678-aa25454287c6 2026-02-23T08:19:32.030 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:32.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCUDZxp5gESAhAA2++yMq4/BK8k3a3I1EaacA== 2026-02-23T08:19:32.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCUDZxp5gESAhAA2++yMq4/BK8k3a3I1EaacA=="}' 2026-02-23T08:19:32.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0ef4ba12-6f58-4c32-b678-aa25454287c6 -i td/test-erasure-eio/6/new.json 2026-02-23T08:19:32.268 INFO:tasks.workunit.client.0.trial186.stdout:6 2026-02-23T08:19:32.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-23T08:19:32.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCUDZxp5gESAhAA2++yMq4/BK8k3a3I1EaacA== --osd-uuid 0ef4ba12-6f58-4c32-b678-aa25454287c6 2026-02-23T08:19:32.297 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:32.288+0000 7f8be5c39a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:32.299 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:32.292+0000 7f8be5c39a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:32.300 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:32.292+0000 7f8be5c39a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:32.301 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:32.292+0000 7f8be5c39a40 -1 bdev(0x555b9cdac400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:32.301 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:32.292+0000 7f8be5c39a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-23T08:19:33.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-23T08:19:33.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:33.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-23T08:19:33.006 INFO:tasks.workunit.client.0.trial186.stdout:adding osd6 key to auth repository 2026-02-23T08:19:33.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:33.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-23T08:19:33.306 INFO:tasks.workunit.client.0.trial186.stdout:start osd.6 2026-02-23T08:19:33.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:33.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:33.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:33.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:33.324 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:33.316+0000 7f3e103dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:33.326 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:33.320+0000 7f3e103dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:33.327 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:33.320+0000 7f3e103dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:33.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:19:33.683 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:33.676+0000 7f3e103dfa40 -1 Falling back to public interface 2026-02-23T08:19:33.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:34.325 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:34.316+0000 7f3e103dfa40 -1 osd.6 0 log_to_monitors true 2026-02-23T08:19:34.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:34.747 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:34.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:34.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:34.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:34.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:19:34.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:35.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:35.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:35.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:19:35.970 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:35.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:35.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:19:36.185 INFO:tasks.workunit.client.0.trial186.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3210780795,v1:127.0.0.1:6851/3210780795] [v2:127.0.0.1:6852/3210780795,v1:127.0.0.1:6853/3210780795] exists,up 0ef4ba12-6f58-4c32-b678-aa25454287c6 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:36.187 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:36.188 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:36.188 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:19:36.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:19:36.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:19:36.243 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:19:36.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:19:36.253 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:19:11.680+0000 7f30a1dc1a40 0 load: jerasure load: lrc 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-23T08:19:36.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:19:36.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-23T08:19:36.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:19:36.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:19:36.890 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:19:36.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:19:37.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:19:37.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:19:37.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:19:37.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:19:37.901 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:19:37.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:19:37.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:19:37.903 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:19:37.903 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:19:37.903 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:19:38.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:19:38.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:19:38.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:19:38.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:19:38.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:19:38.010 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:19:38.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:38.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:19:38.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-23T08:19:38.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-23T08:19:38.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783' 2026-02-23T08:19:38.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:38.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:19:38.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-23T08:19:38.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-23T08:19:38.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855' 2026-02-23T08:19:38.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:38.382 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:19:38.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-23T08:19:38.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-23T08:19:38.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334' 2026-02-23T08:19:38.462 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:38.462 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:19:38.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182405 2026-02-23T08:19:38.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182405 2026-02-23T08:19:38.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-107374182405' 2026-02-23T08:19:38.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:38.542 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:19:38.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953477 2026-02-23T08:19:38.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953477 2026-02-23T08:19:38.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-107374182405 4-137438953477' 2026-02-23T08:19:38.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:38.620 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:19:38.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659140 2026-02-23T08:19:38.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659140 2026-02-23T08:19:38.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-107374182405 4-137438953477 5-176093659140' 2026-02-23T08:19:38.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:38.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:19:38.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-23T08:19:38.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-23T08:19:38.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-107374182405 4-137438953477 5-176093659140 6-210453397507' 2026-02-23T08:19:38.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:38.783 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803783 2026-02-23T08:19:38.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:38.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:19:38.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-23T08:19:38.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:38.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803783 2026-02-23T08:19:38.787 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803783 2026-02-23T08:19:38.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-23T08:19:38.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:19:38.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-23T08:19:39.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:19:40.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:19:40.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:19:40.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803783 2026-02-23T08:19:40.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:40.217 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574855 2026-02-23T08:19:40.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:40.218 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:19:40.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-23T08:19:40.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:40.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-23T08:19:40.221 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574855 2026-02-23T08:19:40.221 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-23T08:19:40.221 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:19:40.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-23T08:19:40.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:40.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411334 2026-02-23T08:19:40.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:40.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:19:40.454 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411334 2026-02-23T08:19:40.454 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:40.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-23T08:19:40.455 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411334 2026-02-23T08:19:40.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-23T08:19:40.456 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:19:40.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-23T08:19:40.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:40.679 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182405 2026-02-23T08:19:40.680 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:40.681 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:19:40.682 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182405 2026-02-23T08:19:40.682 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:40.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182405 2026-02-23T08:19:40.683 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182405 2026-02-23T08:19:40.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-23T08:19:40.684 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:19:40.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182406 -lt 107374182405 2026-02-23T08:19:40.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:40.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-137438953477 2026-02-23T08:19:40.912 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:40.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:19:40.913 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-137438953477 2026-02-23T08:19:40.914 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:40.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=137438953477 2026-02-23T08:19:40.915 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 137438953477 2026-02-23T08:19:40.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 137438953477' 2026-02-23T08:19:40.916 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:19:41.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 137438953477 -lt 137438953477 2026-02-23T08:19:41.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:41.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-176093659140 2026-02-23T08:19:41.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:41.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:19:41.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-176093659140 2026-02-23T08:19:41.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:41.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659140 2026-02-23T08:19:41.140 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 176093659140 2026-02-23T08:19:41.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 176093659140' 2026-02-23T08:19:41.140 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:19:41.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659140 -lt 176093659140 2026-02-23T08:19:41.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:41.366 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397507 2026-02-23T08:19:41.366 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:41.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:19:41.368 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397507 2026-02-23T08:19:41.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:41.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397507 2026-02-23T08:19:41.370 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 210453397507 2026-02-23T08:19:41.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-23T08:19:41.371 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:19:41.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397508 -lt 210453397507 2026-02-23T08:19:41.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:19:41.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:41.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:41.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:19:41.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:19:41.889 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:19:41.889 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:19:41.889 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:19:41.889 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:19:41.890 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:19:41.890 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:19:42.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:19:42.103 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:19:42.104 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:42.104 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:19:42.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-23T08:19:42.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-23T08:19:42.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:19:42.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:19:42.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:19:42.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-23T08:19:42.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-23T08:19:42.722 INFO:tasks.workunit.client.0.trial186.stderr:osd.0 is already out. 2026-02-23T08:19:42.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-23T08:19:42.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:19:42.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:19:42.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:19:42.733 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:19:42.733 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:19:42.733 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:19:42.733 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:19:42.733 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:19:42.734 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:19:42.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:19:42.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:19:42.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:19:42.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:19:42.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:19:42.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:19:43.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:19:43.064 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:19:43.064 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:19:43.064 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:19:43.064 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:19:43.065 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:19:43.065 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:19:43.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:19:43.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:43.065 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:19:43.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-23T08:19:43.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-23T08:19:43.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786' 2026-02-23T08:19:43.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:43.144 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:19:43.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-23T08:19:43.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-23T08:19:43.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858' 2026-02-23T08:19:43.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:43.223 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:19:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-23T08:19:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-23T08:19:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337' 2026-02-23T08:19:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:43.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:19:43.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182408 2026-02-23T08:19:43.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182408 2026-02-23T08:19:43.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-107374182408' 2026-02-23T08:19:43.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:43.382 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:19:43.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=137438953480 2026-02-23T08:19:43.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 137438953480 2026-02-23T08:19:43.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-107374182408 4-137438953480' 2026-02-23T08:19:43.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:43.461 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:19:43.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659143 2026-02-23T08:19:43.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659143 2026-02-23T08:19:43.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-107374182408 4-137438953480 5-176093659143' 2026-02-23T08:19:43.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:19:43.538 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:19:43.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-23T08:19:43.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-23T08:19:43.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-107374182408 4-137438953480 5-176093659143 6-210453397510' 2026-02-23T08:19:43.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:43.620 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803786 2026-02-23T08:19:43.620 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:43.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:19:43.622 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-23T08:19:43.622 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:43.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803786 2026-02-23T08:19:43.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-23T08:19:43.624 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803786 2026-02-23T08:19:43.624 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:19:43.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-23T08:19:43.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:43.851 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574858 2026-02-23T08:19:43.851 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:43.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:19:43.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574858 2026-02-23T08:19:43.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574858 2026-02-23T08:19:43.854 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574858 2026-02-23T08:19:43.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-23T08:19:43.855 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:19:44.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-23T08:19:44.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:44.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411337 2026-02-23T08:19:44.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:44.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:19:44.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411337 2026-02-23T08:19:44.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:44.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-23T08:19:44.077 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411337 2026-02-23T08:19:44.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-23T08:19:44.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:19:44.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-23T08:19:44.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:44.292 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182408 2026-02-23T08:19:44.292 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:44.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:19:44.293 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182408 2026-02-23T08:19:44.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:44.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182408 2026-02-23T08:19:44.295 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182408 2026-02-23T08:19:44.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182408' 2026-02-23T08:19:44.296 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:19:44.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182406 -lt 107374182408 2026-02-23T08:19:44.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:19:45.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:19:45.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:19:45.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182409 -lt 107374182408 2026-02-23T08:19:45.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:45.724 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-137438953480 2026-02-23T08:19:45.724 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:45.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:19:45.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-137438953480 2026-02-23T08:19:45.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:45.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=137438953480 2026-02-23T08:19:45.729 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 137438953480 2026-02-23T08:19:45.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 137438953480' 2026-02-23T08:19:45.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:19:45.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 137438953480 -lt 137438953480 2026-02-23T08:19:45.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:45.963 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-176093659143 2026-02-23T08:19:45.963 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:45.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:19:45.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-176093659143 2026-02-23T08:19:45.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:45.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659143 2026-02-23T08:19:45.966 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 176093659143 2026-02-23T08:19:45.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 176093659143' 2026-02-23T08:19:45.966 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:19:46.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659143 -lt 176093659143 2026-02-23T08:19:46.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:19:46.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397510 2026-02-23T08:19:46.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:19:46.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:19:46.189 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397510 2026-02-23T08:19:46.189 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:19:46.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397510 2026-02-23T08:19:46.191 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 210453397510 2026-02-23T08:19:46.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397510' 2026-02-23T08:19:46.191 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:19:46.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-23T08:19:46.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:19:46.415 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:46.415 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:46.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:19:46.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:19:46.707 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:19:46.707 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:19:46.707 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:19:46.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:19:46.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:19:46.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:19:46.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:19:46.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:19:46.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:19:46.924 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:19:47.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:19:47.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:19:47.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:19:47.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-23T08:19:47.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:19:47.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:19:47.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-23T08:19:47.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:19:47.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:19:47.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-23T08:19:47.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:19:47.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:19:47.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-23T08:19:47.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:19:47.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:19:47.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-23T08:19:47.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:19:47.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:19:47.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-23T08:19:47.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:19:47.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:19:47.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-23T08:19:47.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:19:47.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:19:47.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-23T08:19:47.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:19:47.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:19:47.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-23T08:19:47.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:19:47.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:19:47.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:19:47.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:19:47.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:19:47.561 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:19:47.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:19:47.839 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:19:47.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:19:47.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:19:47.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:19:47.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:19:47.849 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:19:47.849 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:19:47.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:19:47.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:19:47.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:19:48.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:19:48.161 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:19:48.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:19:48.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:19:48.165 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:19:48.165 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:19:48.165 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:19:48.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:48.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:19:48.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:19:48.167 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:48.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:19:48.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:19:48.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:19:48.191 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:19:48.191 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:48.191 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:48.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:19:48.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:19:48.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:19:48.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:19:48.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:19:48.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:19:48.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:19:48.197 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:19:48.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:19:48.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:19:48.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:19:48.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:19:48.199 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:19:48.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:48.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:19:48.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:19:48.201 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:19:48.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:19:48.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:19:48.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:19:48.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:19:48.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:48.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:48.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:19:48.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:19:48.206 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:19:48.206 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:19:48.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:19:48.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:48.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:48.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:19:48.209 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:19:48.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:19:48.233 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:19:48.233 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:48.233 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:48.233 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:48.233 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:48.233 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:48.233 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:48.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:19:48.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:19:48.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:19:48.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:19:48.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:19:48.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:19:48.264 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:19:48.264 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:19:48.264 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:19:48.264 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:19:48.264 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:48.264 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:48.264 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:48.265 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:19:48.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:19:48.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:19:48.324 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:48.325 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:19:48.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:19:48.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:19:48.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:19:48.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:19:48.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:19:48.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:19:48.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:19:48.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:19:48.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:19:48.497 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:19:48.497 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:48.497 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:48.498 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:48.498 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:48.498 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:48.498 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:48.498 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:19:48.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:19:48.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:19:48.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:19:48.623 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:19:48.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:19:49.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:19:49.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:19:49.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:19:49.635 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:19:49.635 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:49.635 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:49.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:19:49.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:19:49.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:19:49.686 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:19:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:19:49.694 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:19:48.247+0000 7f6012e34ec0 0 load: jerasure load: lrc 2026-02-23T08:19:49.694 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:19:49.695 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-23T08:19:49.696 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-23T08:19:49.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:49.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:19:49.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:49.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:49.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:19:49.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:49.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:49.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:19:49.700 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:49.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=660033d0-d863-4405-b8b9-5b4ba00a9b35 2026-02-23T08:19:49.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 660033d0-d863-4405-b8b9-5b4ba00a9b35' 2026-02-23T08:19:49.701 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 660033d0-d863-4405-b8b9-5b4ba00a9b35 2026-02-23T08:19:49.701 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:49.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQClDZxpR5RiKhAABeQqmA3KEh+OGd4iQ3dDGA== 2026-02-23T08:19:49.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQClDZxpR5RiKhAABeQqmA3KEh+OGd4iQ3dDGA=="}' 2026-02-23T08:19:49.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 660033d0-d863-4405-b8b9-5b4ba00a9b35 -i td/test-erasure-eio/0/new.json 2026-02-23T08:19:49.832 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:49.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:19:49.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQClDZxpR5RiKhAABeQqmA3KEh+OGd4iQ3dDGA== --osd-uuid 660033d0-d863-4405-b8b9-5b4ba00a9b35 2026-02-23T08:19:49.859 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:49.855+0000 7fe847a10a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:49.860 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:49.855+0000 7fe847a10a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:49.861 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:49.855+0000 7fe847a10a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:49.861 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:49.855+0000 7fe847a10a40 -1 bdev(0x558166bc8400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:49.862 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:49.859+0000 7fe847a10a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:19:50.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:19:50.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:50.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:19:50.643 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:19:50.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:50.796 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:19:50.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:19:50.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:50.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:50.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:50.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:50.813 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:50.807+0000 7feb9203ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:50.814 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:50.811+0000 7feb9203ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:50.814 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:50.811+0000 7feb9203ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:50.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:50.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:50.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:19:51.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:51.268 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:51.263+0000 7feb9203ea40 -1 Falling back to public interface 2026-02-23T08:19:51.875 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:51.871+0000 7feb9203ea40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:19:52.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:52.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:52.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:52.127 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:52.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:52.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:19:52.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:53.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:53.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:53.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:19:53.352 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:53.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:53.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:19:53.562 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2140576234,v1:127.0.0.1:6803/2140576234] [v2:127.0.0.1:6804/2140576234,v1:127.0.0.1:6805/2140576234] exists,up 660033d0-d863-4405-b8b9-5b4ba00a9b35 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:19:53.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:53.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:53.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:19:53.566 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:53.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4a737d46-83fc-4ab8-8509-3b8992de8068 2026-02-23T08:19:53.567 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 4a737d46-83fc-4ab8-8509-3b8992de8068 2026-02-23T08:19:53.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4a737d46-83fc-4ab8-8509-3b8992de8068' 2026-02-23T08:19:53.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:53.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCpDZxpS1eAIhAANOGN2psMaVMh/FSCc2PrXA== 2026-02-23T08:19:53.582 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCpDZxpS1eAIhAANOGN2psMaVMh/FSCc2PrXA=="}' 2026-02-23T08:19:53.582 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4a737d46-83fc-4ab8-8509-3b8992de8068 -i td/test-erasure-eio/1/new.json 2026-02-23T08:19:53.810 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:53.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:19:53.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCpDZxpS1eAIhAANOGN2psMaVMh/FSCc2PrXA== --osd-uuid 4a737d46-83fc-4ab8-8509-3b8992de8068 2026-02-23T08:19:53.846 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:53.839+0000 7fd0c6b00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:53.847 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:53.843+0000 7fd0c6b00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:53.848 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:53.843+0000 7fd0c6b00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:53.849 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:53.843+0000 7fd0c6b00a40 -1 bdev(0x55594c30c400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:53.849 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:53.843+0000 7fd0c6b00a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:19:54.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:19:54.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:54.538 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:19:54.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:19:54.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:54.840 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:19:54.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:19:54.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:54.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:54.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:54.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:54.857 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:54.851+0000 7f91f71eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:54.858 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:54.855+0000 7f91f71eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:54.860 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:54.855+0000 7f91f71eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:55.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:19:55.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:55.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:19:55.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:55.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:55.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:55.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:55.060 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:55.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:55.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:19:55.236 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:55.231+0000 7f91f71eba40 -1 Falling back to public interface 2026-02-23T08:19:55.285 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:55.880 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:55.875+0000 7f91f71eba40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:19:56.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:56.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:56.287 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:56.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:56.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:56.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:19:56.504 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/552607884,v1:127.0.0.1:6811/552607884] [v2:127.0.0.1:6812/552607884,v1:127.0.0.1:6813/552607884] exists,up 4a737d46-83fc-4ab8-8509-3b8992de8068 2026-02-23T08:19:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:19:56.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:19:56.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:56.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:19:56.511 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:56.512 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ee450199-5c52-4ca5-83d3-f4eda87f5c8f 2026-02-23T08:19:56.512 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 ee450199-5c52-4ca5-83d3-f4eda87f5c8f 2026-02-23T08:19:56.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 ee450199-5c52-4ca5-83d3-f4eda87f5c8f' 2026-02-23T08:19:56.513 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:56.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCsDZxp4qfxHhAAW4iTgbEjwf/PTs780IPX5A== 2026-02-23T08:19:56.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCsDZxp4qfxHhAAW4iTgbEjwf/PTs780IPX5A=="}' 2026-02-23T08:19:56.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ee450199-5c52-4ca5-83d3-f4eda87f5c8f -i td/test-erasure-eio/2/new.json 2026-02-23T08:19:56.742 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:19:56.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:19:56.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCsDZxp4qfxHhAAW4iTgbEjwf/PTs780IPX5A== --osd-uuid ee450199-5c52-4ca5-83d3-f4eda87f5c8f 2026-02-23T08:19:56.780 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:56.775+0000 7f736fa95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:56.782 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:56.775+0000 7f736fa95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:56.783 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:56.779+0000 7f736fa95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:56.784 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:56.779+0000 7f736fa95a40 -1 bdev(0x555aefb72400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:56.784 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:56.779+0000 7f736fa95a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:19:57.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:19:57.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:19:57.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:19:57.423 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:19:57.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:19:57.735 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:19:57.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:19:57.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:19:57.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:19:57.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:19:57.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:19:57.753 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:57.747+0000 7f07486d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:57.755 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:57.751+0000 7f07486d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:57.756 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:57.751+0000 7f07486d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:57.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:19:58.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:19:58.215 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:58.211+0000 7f07486d7a40 -1 Falling back to public interface 2026-02-23T08:19:58.781 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:58.775+0000 7f07486d7a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:19:59.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:19:59.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:19:59.181 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:19:59.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:19:59.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:19:59.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:19:59.397 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4151839134,v1:127.0.0.1:6819/4151839134] [v2:127.0.0.1:6820/4151839134,v1:127.0.0.1:6821/4151839134] exists,up ee450199-5c52-4ca5-83d3-f4eda87f5c8f 2026-02-23T08:19:59.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:19:59.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:19:59.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:19:59.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:19:59.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:19:59.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:19:59.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:19:59.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:19:59.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:19:59.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:19:59.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:19:59.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e4720afd-df08-455e-9c92-1108222b4dd1 2026-02-23T08:19:59.404 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 e4720afd-df08-455e-9c92-1108222b4dd1 2026-02-23T08:19:59.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e4720afd-df08-455e-9c92-1108222b4dd1' 2026-02-23T08:19:59.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:19:59.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCvDZxpKqyKGBAA0NAM+VgxvFflAA0GUt12LQ== 2026-02-23T08:19:59.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCvDZxpKqyKGBAA0NAM+VgxvFflAA0GUt12LQ=="}' 2026-02-23T08:19:59.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e4720afd-df08-455e-9c92-1108222b4dd1 -i td/test-erasure-eio/3/new.json 2026-02-23T08:19:59.626 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:19:59.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:19:59.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCvDZxpKqyKGBAA0NAM+VgxvFflAA0GUt12LQ== --osd-uuid e4720afd-df08-455e-9c92-1108222b4dd1 2026-02-23T08:19:59.654 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:59.647+0000 7f6020e90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:59.655 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:59.651+0000 7f6020e90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:59.656 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:59.651+0000 7f6020e90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:19:59.657 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:59.651+0000 7f6020e90a40 -1 bdev(0x55e81e1b2400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:19:59.657 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:19:59.651+0000 7f6020e90a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:20:00.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:20:00.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:00.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:20:00.294 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:20:00.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:00.591 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:20:00.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:20:00.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:00.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:00.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:00.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:00.617 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:00.611+0000 7f80f6a70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:00.618 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:00.615+0000 7f80f6a70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:00.619 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:00.615+0000 7f80f6a70a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:00.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:00.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:20:00.979 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:00.975+0000 7f80f6a70a40 -1 Falling back to public interface 2026-02-23T08:20:01.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:01.584 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:01.579+0000 7f80f6a70a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:20:02.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:02.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:02.030 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:02.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:02.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:02.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:20:02.250 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:03.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:03.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:03.252 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:20:03.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:20:03.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:03.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:20:03.470 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1364210130,v1:127.0.0.1:6827/1364210130] [v2:127.0.0.1:6828/1364210130,v1:127.0.0.1:6829/1364210130] exists,up e4720afd-df08-455e-9c92-1108222b4dd1 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:20:03.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:03.478 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:03.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:03.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:03.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:03.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-23T08:20:03.481 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:03.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4c2f68bf-291c-4d86-8fb9-b911f007be36 2026-02-23T08:20:03.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 4c2f68bf-291c-4d86-8fb9-b911f007be36' 2026-02-23T08:20:03.481 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 4c2f68bf-291c-4d86-8fb9-b911f007be36 2026-02-23T08:20:03.482 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:03.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCzDZxpJfbvHBAAW9dbSikcQdmO3uYN2QJDbQ== 2026-02-23T08:20:03.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCzDZxpJfbvHBAAW9dbSikcQdmO3uYN2QJDbQ=="}' 2026-02-23T08:20:03.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4c2f68bf-291c-4d86-8fb9-b911f007be36 -i td/test-erasure-eio/4/new.json 2026-02-23T08:20:03.707 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:20:03.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-23T08:20:03.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCzDZxpJfbvHBAAW9dbSikcQdmO3uYN2QJDbQ== --osd-uuid 4c2f68bf-291c-4d86-8fb9-b911f007be36 2026-02-23T08:20:03.737 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:03.731+0000 7f8bf5998a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:03.738 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:03.731+0000 7f8bf5998a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:03.739 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:03.735+0000 7f8bf5998a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:03.740 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:03.735+0000 7f8bf5998a40 -1 bdev(0x561c1bbcc400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:03.740 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:03.735+0000 7f8bf5998a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-23T08:20:04.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-23T08:20:04.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:04.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:20:04.334 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:20:04.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:04.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:20:04.639 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:20:04.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:04.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:04.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:04.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:04.657 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:04.651+0000 7f233b196a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:04.658 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:04.655+0000 7f233b196a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:04.659 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:04.655+0000 7f233b196a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:04.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:20:04.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:04.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:20:04.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:04.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:04.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:04.852 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:04.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:04.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:04.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:20:05.059 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:05.055+0000 7f233b196a40 -1 Falling back to public interface 2026-02-23T08:20:05.070 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:05.656 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:05.651+0000 7f233b196a40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:20:06.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:06.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:06.071 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:06.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:06.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:06.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:20:06.289 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2745529769,v1:127.0.0.1:6835/2745529769] [v2:127.0.0.1:6836/2745529769,v1:127.0.0.1:6837/2745529769] exists,up 4c2f68bf-291c-4d86-8fb9-b911f007be36 2026-02-23T08:20:06.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-23T08:20:06.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:06.294 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:06.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:06.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:06.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:06.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:06.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:06.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-23T08:20:06.296 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:06.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d3e2f34e-8a2c-47a7-895f-fd78b168f792 2026-02-23T08:20:06.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 d3e2f34e-8a2c-47a7-895f-fd78b168f792' 2026-02-23T08:20:06.297 INFO:tasks.workunit.client.0.trial186.stdout:add osd5 d3e2f34e-8a2c-47a7-895f-fd78b168f792 2026-02-23T08:20:06.298 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:06.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC2DZxpMHQPEhAAXdNIjrHzvOO36uSpodEOTg== 2026-02-23T08:20:06.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC2DZxpMHQPEhAAXdNIjrHzvOO36uSpodEOTg=="}' 2026-02-23T08:20:06.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d3e2f34e-8a2c-47a7-895f-fd78b168f792 -i td/test-erasure-eio/5/new.json 2026-02-23T08:20:06.526 INFO:tasks.workunit.client.0.trial186.stdout:5 2026-02-23T08:20:06.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-23T08:20:06.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC2DZxpMHQPEhAAXdNIjrHzvOO36uSpodEOTg== --osd-uuid d3e2f34e-8a2c-47a7-895f-fd78b168f792 2026-02-23T08:20:06.555 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:06.551+0000 7ff3e4065a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:06.556 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:06.551+0000 7ff3e4065a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:06.557 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:06.551+0000 7ff3e4065a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:06.558 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:06.551+0000 7ff3e4065a40 -1 bdev(0x5623441a8400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:06.558 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:06.555+0000 7ff3e4065a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-23T08:20:07.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-23T08:20:07.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:07.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-23T08:20:07.243 INFO:tasks.workunit.client.0.trial186.stdout:adding osd5 key to auth repository 2026-02-23T08:20:07.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:07.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-23T08:20:07.535 INFO:tasks.workunit.client.0.trial186.stdout:start osd.5 2026-02-23T08:20:07.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:07.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:07.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:07.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:07.553 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:07.547+0000 7fac5be25a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:07.554 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:07.551+0000 7fac5be25a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:07.555 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:07.551+0000 7fac5be25a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:07.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-23T08:20:07.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:07.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-23T08:20:07.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:07.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:07.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:07.750 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:07.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:07.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:07.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:20:07.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:07.980 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:07.975+0000 7fac5be25a40 -1 Falling back to public interface 2026-02-23T08:20:08.528 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:08.523+0000 7fac5be25a40 -1 osd.5 0 log_to_monitors true 2026-02-23T08:20:08.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:08.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:08.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:08.966 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:08.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:08.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:20:09.189 INFO:tasks.workunit.client.0.trial186.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2573704788,v1:127.0.0.1:6843/2573704788] [v2:127.0.0.1:6844/2573704788,v1:127.0.0.1:6845/2573704788] exists,up d3e2f34e-8a2c-47a7-895f-fd78b168f792 2026-02-23T08:20:09.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:09.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:20:09.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:09.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:09.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:09.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-23T08:20:09.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:09.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=103b97a5-eaa3-4352-9052-bc89626926da 2026-02-23T08:20:09.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 103b97a5-eaa3-4352-9052-bc89626926da' 2026-02-23T08:20:09.197 INFO:tasks.workunit.client.0.trial186.stdout:add osd6 103b97a5-eaa3-4352-9052-bc89626926da 2026-02-23T08:20:09.197 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:09.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5DZxpWfUfDBAAYUEFUGs7CA/O9J3pHSjv2w== 2026-02-23T08:20:09.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5DZxpWfUfDBAAYUEFUGs7CA/O9J3pHSjv2w=="}' 2026-02-23T08:20:09.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 103b97a5-eaa3-4352-9052-bc89626926da -i td/test-erasure-eio/6/new.json 2026-02-23T08:20:09.421 INFO:tasks.workunit.client.0.trial186.stdout:6 2026-02-23T08:20:09.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-23T08:20:09.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5DZxpWfUfDBAAYUEFUGs7CA/O9J3pHSjv2w== --osd-uuid 103b97a5-eaa3-4352-9052-bc89626926da 2026-02-23T08:20:09.460 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:09.455+0000 7f1e89e7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:09.462 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:09.455+0000 7f1e89e7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:09.463 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:09.459+0000 7f1e89e7da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:09.463 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:09.459+0000 7f1e89e7da40 -1 bdev(0x5588a43b8400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:09.464 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:09.459+0000 7f1e89e7da40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-23T08:20:10.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-23T08:20:10.154 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:10.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-23T08:20:10.155 INFO:tasks.workunit.client.0.trial186.stdout:adding osd6 key to auth repository 2026-02-23T08:20:10.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:10.454 INFO:tasks.workunit.client.0.trial186.stdout:start osd.6 2026-02-23T08:20:10.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-23T08:20:10.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:10.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:10.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:10.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:10.481 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:10.475+0000 7f4f5c34aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:10.483 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:10.479+0000 7f4f5c34aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:10.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:10.479+0000 7f4f5c34aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:10.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-23T08:20:10.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:10.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-23T08:20:10.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:10.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:10.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:10.677 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:10.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:10.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:10.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:20:10.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:10.956 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:10.951+0000 7f4f5c34aa40 -1 Falling back to public interface 2026-02-23T08:20:11.627 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:11.623+0000 7f4f5c34aa40 -1 osd.6 0 log_to_monitors true 2026-02-23T08:20:11.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:11.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:11.896 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:11.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:11.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:11.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:20:12.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:13.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:13.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:13.117 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:20:13.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:20:13.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:13.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:20:13.340 INFO:tasks.workunit.client.0.trial186.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3341022321,v1:127.0.0.1:6851/3341022321] [v2:127.0.0.1:6852/3341022321,v1:127.0.0.1:6853/3341022321] exists,up 103b97a5-eaa3-4352-9052-bc89626926da 2026-02-23T08:20:13.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:13.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:13.342 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:13.342 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:20:13.343 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:20:13.400 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:20:13.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:20:13.409 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:19:51.267+0000 7feb9203ea40 0 load: jerasure load: lrc 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-23T08:20:13.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:20:13.411 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-23T08:20:13.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:20:13.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:20:14.062 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:20:14.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:20:15.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:20:15.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:20:15.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:20:15.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:20:15.075 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:20:15.076 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:20:15.076 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:20:15.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:20:15.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:20:15.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:20:15.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:20:15.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:20:15.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:20:15.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:20:15.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:20:15.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:20:15.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:20:15.394 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:20:15.394 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:20:15.395 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:20:15.395 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:20:15.395 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:20:15.395 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:20:15.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:20:15.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:15.395 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:20:15.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-23T08:20:15.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-23T08:20:15.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-23T08:20:15.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:15.475 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:20:15.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-23T08:20:15.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-23T08:20:15.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607558' 2026-02-23T08:20:15.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:15.556 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:20:15.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444038 2026-02-23T08:20:15.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444038 2026-02-23T08:20:15.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-73014444038' 2026-02-23T08:20:15.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:15.636 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:20:15.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182405 2026-02-23T08:20:15.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182405 2026-02-23T08:20:15.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-73014444038 3-107374182405' 2026-02-23T08:20:15.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:15.717 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:20:15.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920772 2026-02-23T08:20:15.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920772 2026-02-23T08:20:15.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-73014444038 3-107374182405 4-141733920772' 2026-02-23T08:20:15.795 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:15.795 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:20:15.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626436 2026-02-23T08:20:15.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626436 2026-02-23T08:20:15.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-73014444038 3-107374182405 4-141733920772 5-180388626436' 2026-02-23T08:20:15.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:15.876 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:20:15.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-23T08:20:15.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-23T08:20:15.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607558 2-73014444038 3-107374182405 4-141733920772 5-180388626436 6-214748364803' 2026-02-23T08:20:15.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:15.955 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-23T08:20:15.955 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:15.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:20:15.957 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-23T08:20:15.957 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:15.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-23T08:20:15.958 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803784 2026-02-23T08:20:15.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-23T08:20:15.959 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:20:16.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-23T08:20:16.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:20:17.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:20:17.183 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:20:17.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-23T08:20:17.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:17.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-23T08:20:17.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:17.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:20:17.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-23T08:20:17.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:17.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-23T08:20:17.403 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607558 2026-02-23T08:20:17.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-23T08:20:17.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:20:17.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-23T08:20:17.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:17.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444038 2026-02-23T08:20:17.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:17.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:20:17.632 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444038 2026-02-23T08:20:17.632 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:17.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444038 2026-02-23T08:20:17.634 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444038 2026-02-23T08:20:17.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444038' 2026-02-23T08:20:17.634 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:20:17.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444038 -lt 73014444038 2026-02-23T08:20:17.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:17.857 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182405 2026-02-23T08:20:17.857 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:17.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:20:17.858 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182405 2026-02-23T08:20:17.858 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:17.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182405 2026-02-23T08:20:17.860 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182405 2026-02-23T08:20:17.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-23T08:20:17.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:20:18.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-23T08:20:18.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:18.081 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920772 2026-02-23T08:20:18.081 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:18.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:20:18.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920772 2026-02-23T08:20:18.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:18.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920772 2026-02-23T08:20:18.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920772' 2026-02-23T08:20:18.085 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 141733920772 2026-02-23T08:20:18.086 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:20:18.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920773 -lt 141733920772 2026-02-23T08:20:18.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:18.314 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626436 2026-02-23T08:20:18.314 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:18.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:20:18.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626436 2026-02-23T08:20:18.316 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:18.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626436 2026-02-23T08:20:18.317 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 180388626436 2026-02-23T08:20:18.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626436' 2026-02-23T08:20:18.317 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:20:18.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626436 -lt 180388626436 2026-02-23T08:20:18.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:18.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364803 2026-02-23T08:20:18.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:18.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:20:18.554 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364803 2026-02-23T08:20:18.554 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:18.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364803 2026-02-23T08:20:18.555 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 214748364803 2026-02-23T08:20:18.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-23T08:20:18.556 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:20:18.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-23T08:20:18.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:20:18.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:20:18.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:20:19.073 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:20:19.282 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:20:19.283 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:20:19.283 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:20:19.284 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-23T08:20:19.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:20:19.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:19.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:20:19.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:20:19.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:19.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-23T08:20:19.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:20:19.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:20:19.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:19.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:20:19.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:20:19.639 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:20:19.847 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:20:19.848 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-23T08:20:19.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:20:19.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:19.851 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:20:19.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:20:19.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:20:19.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:20:19.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:20:19.915 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-23T08:20:19.915 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:20:19.915 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:20:19.915 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:19.915 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:19.915 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:19.916 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:20:19.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:20:19.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-23T08:20:19.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-23T08:20:20.244 INFO:tasks.workunit.client.0.trial186.stderr:osd.0 is already out. 2026-02-23T08:20:20.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-23T08:20:20.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:20:20.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:20:20.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:20:20.256 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:20:20.256 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:20:20.256 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:20:20.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:20:20.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:20:20.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:20:20.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:20:20.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:20:20.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:20:20.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:20:20.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:20:20.364 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:20:20.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:20.582 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:20:20.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-23T08:20:20.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-23T08:20:20.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-23T08:20:20.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:20.663 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:20:20.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-23T08:20:20.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-23T08:20:20.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607561' 2026-02-23T08:20:20.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:20.742 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:20:20.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444041 2026-02-23T08:20:20.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444041 2026-02-23T08:20:20.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607561 2-73014444041' 2026-02-23T08:20:20.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:20.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:20:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182408 2026-02-23T08:20:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182408 2026-02-23T08:20:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607561 2-73014444041 3-107374182408' 2026-02-23T08:20:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:20.901 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:20:20.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920775 2026-02-23T08:20:20.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920775 2026-02-23T08:20:20.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607561 2-73014444041 3-107374182408 4-141733920775' 2026-02-23T08:20:20.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:20.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:20:21.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626439 2026-02-23T08:20:21.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626439 2026-02-23T08:20:21.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607561 2-73014444041 3-107374182408 4-141733920775 5-180388626439' 2026-02-23T08:20:21.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:21.064 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:20:21.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364806 2026-02-23T08:20:21.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364806 2026-02-23T08:20:21.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-51539607561 2-73014444041 3-107374182408 4-141733920775 5-180388626439 6-214748364806' 2026-02-23T08:20:21.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:21.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-23T08:20:21.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:21.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:20:21.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-23T08:20:21.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:21.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-23T08:20:21.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-23T08:20:21.149 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803787 2026-02-23T08:20:21.149 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:20:21.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-23T08:20:21.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:21.371 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607561 2026-02-23T08:20:21.372 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:21.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:20:21.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-23T08:20:21.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:21.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607561 2026-02-23T08:20:21.375 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607561 2026-02-23T08:20:21.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-23T08:20:21.376 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:20:21.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-23T08:20:21.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:21.595 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444041 2026-02-23T08:20:21.595 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:21.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:20:21.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444041 2026-02-23T08:20:21.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:21.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444041 2026-02-23T08:20:21.598 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444041 2026-02-23T08:20:21.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444041' 2026-02-23T08:20:21.598 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:20:21.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444041 -lt 73014444041 2026-02-23T08:20:21.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:21.819 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182408 2026-02-23T08:20:21.819 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:21.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:20:21.821 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182408 2026-02-23T08:20:21.821 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:21.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182408 2026-02-23T08:20:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182408' 2026-02-23T08:20:21.823 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182408 2026-02-23T08:20:21.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:20:22.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182406 -lt 107374182408 2026-02-23T08:20:22.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:20:23.041 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:20:23.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:20:23.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182409 -lt 107374182408 2026-02-23T08:20:23.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:23.261 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920775 2026-02-23T08:20:23.261 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:23.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:20:23.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920775 2026-02-23T08:20:23.263 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:23.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920775 2026-02-23T08:20:23.264 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 141733920775 2026-02-23T08:20:23.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920775' 2026-02-23T08:20:23.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:20:23.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920776 -lt 141733920775 2026-02-23T08:20:23.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:23.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626439 2026-02-23T08:20:23.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:23.497 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:20:23.498 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626439 2026-02-23T08:20:23.498 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:23.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626439 2026-02-23T08:20:23.500 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 180388626439 2026-02-23T08:20:23.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626439' 2026-02-23T08:20:23.500 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:20:23.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626439 -lt 180388626439 2026-02-23T08:20:23.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:23.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364806 2026-02-23T08:20:23.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:23.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:20:23.730 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364806 2026-02-23T08:20:23.730 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:23.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364806 2026-02-23T08:20:23.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364806' 2026-02-23T08:20:23.732 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 214748364806 2026-02-23T08:20:23.732 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:20:23.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364807 -lt 214748364806 2026-02-23T08:20:23.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:20:23.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:20:23.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:20:24.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:20:24.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:20:24.254 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:20:24.254 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:20:24.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:20:24.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:20:24.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:20:24.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:20:24.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:20:24.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:20:24.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:20:24.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:20:24.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-23T08:20:24.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:20:24.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:20:24.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-23T08:20:24.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:20:24.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:20:24.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-23T08:20:24.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:20:24.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:20:24.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-23T08:20:24.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:20:24.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:20:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-23T08:20:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:20:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:20:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-23T08:20:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:20:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:20:24.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-23T08:20:24.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:20:24.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:20:24.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:20:24.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:20:24.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:20:25.108 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:20:25.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:20:25.391 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:20:25.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:20:25.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:20:25.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:20:25.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:20:25.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:20:25.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:20:25.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:20:25.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:20:25.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:20:25.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:20:25.716 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:20:25.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:20:25.717 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:20:25.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:20:25.719 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:20:25.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:20:25.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:20:25.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:20:25.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:20:25.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:20:25.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:20:25.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:20:25.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:20:25.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:20:25.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:25.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:25.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:20:25.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:20:25.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:20:25.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:20:25.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:20:25.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:20:25.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:20:25.751 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:20:25.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:20:25.752 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:20:25.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:20:25.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:20:25.753 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:20:25.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:20:25.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:20:25.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:20:25.755 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:20:25.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:20:25.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:20:25.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:20:25.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:20:25.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:25.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:25.759 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:20:25.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:20:25.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:20:25.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:20:25.761 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:20:25.761 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:25.761 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:25.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:20:25.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:20:25.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:20:25.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:20:25.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:20:25.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:20:25.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:20:25.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:20:25.764 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:20:25.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:20:25.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:20:25.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:20:25.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:20:25.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:25.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:25.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:25.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:25.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:25.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:25.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:20:25.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:20:25.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:20:25.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:20:25.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:20:25.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:20:25.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:20:25.826 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:20:25.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:20:25.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:20:25.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:20:25.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:20:25.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:20:25.884 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:20:25.885 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:20:25.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:20:25.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:20:25.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:20:25.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:20:25.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:20:25.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:20:25.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:20:26.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:20:26.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:26.055 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:26.055 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:26.055 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:26.055 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:26.056 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:26.056 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:20:26.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:20:26.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:20:26.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:20:26.180 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:20:26.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:20:27.190 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:20:27.190 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:20:27.190 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:20:27.190 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:27.190 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:27.190 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:27.191 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:20:27.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:20:27.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:20:27.241 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:20:27.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:20:27.248 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:20:25.807+0000 7fd30b5c4ec0 0 load: jerasure load: lrc 2026-02-23T08:20:27.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:20:27.249 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-23T08:20:27.250 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:27.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:27.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:20:27.253 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:27.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0eb9a3de-897c-4251-884a-b75a8595eb5b 2026-02-23T08:20:27.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0eb9a3de-897c-4251-884a-b75a8595eb5b' 2026-02-23T08:20:27.255 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 0eb9a3de-897c-4251-884a-b75a8595eb5b 2026-02-23T08:20:27.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:27.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDLDZxpBi3HDxAAt51fq25EEWMM7YPoROwpkg== 2026-02-23T08:20:27.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDLDZxpBi3HDxAAt51fq25EEWMM7YPoROwpkg=="}' 2026-02-23T08:20:27.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0eb9a3de-897c-4251-884a-b75a8595eb5b -i td/test-erasure-eio/0/new.json 2026-02-23T08:20:27.386 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:27.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:20:27.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDLDZxpBi3HDxAAt51fq25EEWMM7YPoROwpkg== --osd-uuid 0eb9a3de-897c-4251-884a-b75a8595eb5b 2026-02-23T08:20:27.415 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:27.411+0000 7f177b3b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:27.416 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:27.411+0000 7f177b3b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:27.417 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:27.411+0000 7f177b3b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:27.417 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:27.411+0000 7f177b3b3a40 -1 bdev(0x55f56e320400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:27.417 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:27.411+0000 7f177b3b3a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:20:28.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:20:28.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:28.158 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:20:28.159 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:20:28.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:28.306 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:20:28.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:20:28.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:28.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:28.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:28.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:28.324 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:28.319+0000 7f97f5c40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:28.326 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:28.319+0000 7f97f5c40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:28.327 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:28.323+0000 7f97f5c40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:28.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:20:28.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:28.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:20:28.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:28.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:28.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:28.422 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:28.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:28.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:28.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:20:28.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:28.752 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:28.747+0000 7f97f5c40a40 -1 Falling back to public interface 2026-02-23T08:20:29.360 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:29.355+0000 7f97f5c40a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:20:29.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:29.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:29.639 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:29.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:29.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:29.640 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:20:29.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:30.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:30.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:30.855 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:20:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:20:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:20:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:31.068 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3945699813,v1:127.0.0.1:6803/3945699813] [v2:127.0.0.1:6804/3945699813,v1:127.0.0.1:6805/3945699813] exists,up 0eb9a3de-897c-4251-884a-b75a8595eb5b 2026-02-23T08:20:31.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:31.072 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:31.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:31.074 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:31.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:31.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:20:31.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:31.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7e65d415-bca1-4280-8854-c1a3b0294d29 2026-02-23T08:20:31.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7e65d415-bca1-4280-8854-c1a3b0294d29' 2026-02-23T08:20:31.077 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 7e65d415-bca1-4280-8854-c1a3b0294d29 2026-02-23T08:20:31.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:31.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDPDZxp72EHBRAApdPZNOM9tR9VTacmGDPLOw== 2026-02-23T08:20:31.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDPDZxp72EHBRAApdPZNOM9tR9VTacmGDPLOw=="}' 2026-02-23T08:20:31.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7e65d415-bca1-4280-8854-c1a3b0294d29 -i td/test-erasure-eio/1/new.json 2026-02-23T08:20:31.311 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:31.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:20:31.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDPDZxp72EHBRAApdPZNOM9tR9VTacmGDPLOw== --osd-uuid 7e65d415-bca1-4280-8854-c1a3b0294d29 2026-02-23T08:20:31.341 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:31.335+0000 7f0c2c946a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:31.343 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:31.339+0000 7f0c2c946a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:31.343 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:31.339+0000 7f0c2c946a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:31.344 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:31.339+0000 7f0c2c946a40 -1 bdev(0x55db004da400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:31.344 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:31.339+0000 7f0c2c946a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:20:31.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:20:31.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:31.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:20:31.953 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:20:31.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:32.248 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:20:32.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:20:32.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:32.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:32.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:32.249 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:32.277 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:32.271+0000 7fca1e3a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:32.279 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:32.275+0000 7fca1e3a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:32.280 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:32.275+0000 7fca1e3a1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:32.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:20:32.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:32.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:20:32.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:32.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:32.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:32.476 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:32.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:32.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:32.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:20:32.636 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:32.631+0000 7fca1e3a1a40 -1 Falling back to public interface 2026-02-23T08:20:32.694 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:33.308 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:33.303+0000 7fca1e3a1a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:20:33.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:33.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:33.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:33.696 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:33.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:33.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:20:33.915 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:34.917 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:34.917 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:34.918 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:20:34.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:20:34.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:34.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:20:35.132 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3773952645,v1:127.0.0.1:6811/3773952645] [v2:127.0.0.1:6812/3773952645,v1:127.0.0.1:6813/3773952645] exists,up 7e65d415-bca1-4280-8854-c1a3b0294d29 2026-02-23T08:20:35.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:35.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:35.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:35.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:35.137 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:35.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:35.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:20:35.140 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:35.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8cecc3c0-792f-4301-a978-dc84d3f824ae 2026-02-23T08:20:35.140 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 8cecc3c0-792f-4301-a978-dc84d3f824ae 2026-02-23T08:20:35.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 8cecc3c0-792f-4301-a978-dc84d3f824ae' 2026-02-23T08:20:35.141 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:35.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDTDZxpLgPcCBAA4+G+SY5/C2MNLdtRr9aPIA== 2026-02-23T08:20:35.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDTDZxpLgPcCBAA4+G+SY5/C2MNLdtRr9aPIA=="}' 2026-02-23T08:20:35.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8cecc3c0-792f-4301-a978-dc84d3f824ae -i td/test-erasure-eio/2/new.json 2026-02-23T08:20:35.384 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:20:35.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:20:35.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDTDZxpLgPcCBAA4+G+SY5/C2MNLdtRr9aPIA== --osd-uuid 8cecc3c0-792f-4301-a978-dc84d3f824ae 2026-02-23T08:20:35.424 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:35.419+0000 7fe171881a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:35.425 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:35.419+0000 7fe171881a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:35.426 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:35.423+0000 7fe171881a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:35.427 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:35.423+0000 7fe171881a40 -1 bdev(0x5563b8fec400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:35.427 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:35.423+0000 7fe171881a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:20:36.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:20:36.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:36.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:20:36.019 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:20:36.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:36.311 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:20:36.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:20:36.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:36.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:36.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:36.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:36.330 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:36.323+0000 7f3299e14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:36.331 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:36.327+0000 7f3299e14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:36.332 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:36.327+0000 7f3299e14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:36.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:20:36.542 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:36.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:20:36.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:36.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:36.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:36.543 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:36.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:36.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:36.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:20:36.684 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:36.679+0000 7f3299e14a40 -1 Falling back to public interface 2026-02-23T08:20:36.759 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:37.318 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:37.311+0000 7f3299e14a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:20:37.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:37.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:37.760 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:37.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:37.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:37.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:20:37.977 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2381494729,v1:127.0.0.1:6819/2381494729] [v2:127.0.0.1:6820/2381494729,v1:127.0.0.1:6821/2381494729] exists,up 8cecc3c0-792f-4301-a978-dc84d3f824ae 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:20:37.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:37.982 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:37.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:37.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:37.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:37.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:37.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:37.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:20:37.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:37.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=59897f7a-4de4-4481-9068-466986fb1422 2026-02-23T08:20:37.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 59897f7a-4de4-4481-9068-466986fb1422' 2026-02-23T08:20:37.985 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 59897f7a-4de4-4481-9068-466986fb1422 2026-02-23T08:20:37.985 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:37.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDVDZxp7wIuOxAAaSzTQl+SmWEA8nCHXV9WHQ== 2026-02-23T08:20:37.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDVDZxp7wIuOxAAaSzTQl+SmWEA8nCHXV9WHQ=="}' 2026-02-23T08:20:37.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 59897f7a-4de4-4481-9068-466986fb1422 -i td/test-erasure-eio/3/new.json 2026-02-23T08:20:38.220 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:20:38.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:20:38.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDVDZxp7wIuOxAAaSzTQl+SmWEA8nCHXV9WHQ== --osd-uuid 59897f7a-4de4-4481-9068-466986fb1422 2026-02-23T08:20:38.251 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:38.243+0000 7f8d3ea3ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:38.252 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:38.247+0000 7f8d3ea3ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:38.253 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:38.247+0000 7f8d3ea3ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:38.253 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:38.247+0000 7f8d3ea3ba40 -1 bdev(0x564e33006400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:38.254 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:38.247+0000 7f8d3ea3ba40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:20:38.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:20:38.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:38.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:20:38.875 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:20:38.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:39.171 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:20:39.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:20:39.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:39.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:39.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:39.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:39.199 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:39.194+0000 7f9252154a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:39.201 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:39.194+0000 7f9252154a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:39.202 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:39.194+0000 7f9252154a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:39.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:20:39.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:39.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:20:39.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:39.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:39.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:39.393 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:39.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:39.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:39.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:20:39.613 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:39.652 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:39.646+0000 7f9252154a40 -1 Falling back to public interface 2026-02-23T08:20:40.293 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:40.286+0000 7f9252154a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:20:40.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:40.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:40.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:40.616 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:40.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:40.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:20:40.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:41.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:41.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:41.840 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:20:41.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:20:41.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:41.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:20:42.054 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/109274233,v1:127.0.0.1:6827/109274233] [v2:127.0.0.1:6828/109274233,v1:127.0.0.1:6829/109274233] exists,up 59897f7a-4de4-4481-9068-466986fb1422 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:20:42.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:42.059 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:42.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:42.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:42.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:42.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:42.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:20:42.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:20:42.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-23T08:20:42.061 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:20:42.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=35b72df3-9280-4109-b09e-df6197b1269a 2026-02-23T08:20:42.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 35b72df3-9280-4109-b09e-df6197b1269a' 2026-02-23T08:20:42.062 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 35b72df3-9280-4109-b09e-df6197b1269a 2026-02-23T08:20:42.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:20:42.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDaDZxp7Ao8BBAAMGxJIk6mnISnAZs0OG2FCw== 2026-02-23T08:20:42.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDaDZxp7Ao8BBAAMGxJIk6mnISnAZs0OG2FCw=="}' 2026-02-23T08:20:42.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 35b72df3-9280-4109-b09e-df6197b1269a -i td/test-erasure-eio/4/new.json 2026-02-23T08:20:42.290 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:20:42.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-23T08:20:42.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDaDZxp7Ao8BBAAMGxJIk6mnISnAZs0OG2FCw== --osd-uuid 35b72df3-9280-4109-b09e-df6197b1269a 2026-02-23T08:20:42.319 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:42.314+0000 7f47e3f47a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:42.320 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:42.314+0000 7f47e3f47a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:42.321 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:42.314+0000 7f47e3f47a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:42.322 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:42.314+0000 7f47e3f47a40 -1 bdev(0x565425c9c400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:20:42.322 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:42.314+0000 7f47e3f47a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-23T08:20:42.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-23T08:20:42.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:20:42.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:20:42.954 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:20:42.955 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:20:43.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:20:43.264 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:20:43.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:43.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:20:43.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:20:43.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:20:43.283 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:43.278+0000 7fb521387a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:43.284 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:43.278+0000 7fb521387a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:43.285 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:43.278+0000 7fb521387a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:43.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:20:43.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:43.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:20:43.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:43.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:43.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:43.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:43.485 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:43.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:43.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:20:43.688 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:43.682+0000 7fb521387a40 -1 Falling back to public interface 2026-02-23T08:20:43.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:44.299 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:44.294+0000 7fb521387a40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:20:44.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:44.701 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:44.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:44.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:44.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:44.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:20:44.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:45.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:45.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:45.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:20:45.922 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:20:45.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:45.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:20:46.141 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3080159353,v1:127.0.0.1:6835/3080159353] [v2:127.0.0.1:6836/3080159353,v1:127.0.0.1:6837/3080159353] exists,up 35b72df3-9280-4109-b09e-df6197b1269a 2026-02-23T08:20:46.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:46.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:20:46.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:20:46.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:20:46.199 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:20:46.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:20:46.207 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:20:28.751+0000 7f97f5c40a40 0 load: jerasure load: lrc 2026-02-23T08:20:46.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:20:46.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-23T08:20:46.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-23T08:20:46.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:20:46.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:20:46.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-23T08:20:46.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:20:46.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-23T08:20:46.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:20:46.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-23T08:20:46.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:20:46.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:20:46.855 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:20:46.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:20:47.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:20:47.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:20:47.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:20:47.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:20:47.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:20:47.868 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:20:47.868 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:20:47.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:20:47.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:20:47.869 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:20:47.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:20:47.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:20:47.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:20:47.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:20:47.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:20:47.983 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:20:48.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:20:48.197 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:20:48.197 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:20:48.197 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:20:48.198 INFO:tasks.workunit.client.0.trial186.stderr:4' 2026-02-23T08:20:48.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:20:48.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:48.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:20:48.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-23T08:20:48.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-23T08:20:48.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-23T08:20:48.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:48.278 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:20:48.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-23T08:20:48.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-23T08:20:48.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853' 2026-02-23T08:20:48.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:48.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:20:48.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-23T08:20:48.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-23T08:20:48.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-77309411333' 2026-02-23T08:20:48.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:48.437 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:20:48.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182404 2026-02-23T08:20:48.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182404 2026-02-23T08:20:48.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-77309411333 3-107374182404' 2026-02-23T08:20:48.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:20:48.517 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:20:48.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888067 2026-02-23T08:20:48.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888067 2026-02-23T08:20:48.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574853 2-77309411333 3-107374182404 4-146028888067' 2026-02-23T08:20:48.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:48.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-23T08:20:48.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:48.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:20:48.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-23T08:20:48.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:48.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-23T08:20:48.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-23T08:20:48.599 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803782 2026-02-23T08:20:48.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:20:48.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-23T08:20:48.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:48.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-23T08:20:48.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:48.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:20:48.814 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-23T08:20:48.814 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:48.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-23T08:20:48.816 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574853 2026-02-23T08:20:48.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-23T08:20:48.817 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:20:49.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574851 -lt 55834574853 2026-02-23T08:20:49.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:20:50.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:20:50.032 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:20:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574851 -lt 55834574853 2026-02-23T08:20:50.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:20:51.247 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-23T08:20:51.248 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:20:51.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574853 2026-02-23T08:20:51.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:51.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-23T08:20:51.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:51.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:20:51.473 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-23T08:20:51.473 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:51.474 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-23T08:20:51.475 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411333 2026-02-23T08:20:51.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-23T08:20:51.476 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:20:51.705 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411333 2026-02-23T08:20:51.706 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:51.706 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182404 2026-02-23T08:20:51.706 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:51.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:20:51.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182404 2026-02-23T08:20:51.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:51.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182404 2026-02-23T08:20:51.710 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182404 2026-02-23T08:20:51.710 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182404' 2026-02-23T08:20:51.710 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:20:51.932 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-23T08:20:51.932 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:20:51.933 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888067 2026-02-23T08:20:51.933 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:20:51.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:20:51.935 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888067 2026-02-23T08:20:51.935 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:20:51.936 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888067 2026-02-23T08:20:51.936 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 146028888067 2026-02-23T08:20:51.937 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888067' 2026-02-23T08:20:51.937 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:20:52.157 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888067 -lt 146028888067 2026-02-23T08:20:52.158 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:20:52.158 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:20:52.158 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:20:52.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:20:52.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:20:52.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:20:52.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:20:52.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:20:52.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:20:52.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:20:52.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:20:52.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:20:52.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:20:52.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:20:52.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:20:52.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:20:52.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:20:52.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:20:52.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-23T08:20:53.159 INFO:tasks.workunit.client.0.trial186.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-23T08:20:53.159 INFO:tasks.workunit.client.0.trial186.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:47.296548+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-23T08:20:46.799275+0000 0'0 2026-02-23T08:20:46.799275+0000 0 0 periodic scrub scheduled @ 2026-02-24T18:31:15.062015+0000 0 0 2026-02-23T08:20:53.159 INFO:tasks.workunit.client.0.trial186.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.363223+0000 0'0 41:20 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T17:21:52.834667+0000 0 0 2026-02-23T08:20:53.159 INFO:tasks.workunit.client.0.trial186.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.334336+0000 0'0 41:94 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:35:02.623196+0000 0 0 2026-02-23T08:20:53.159 INFO:tasks.workunit.client.0.trial186.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.363715+0000 0'0 41:20 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T16:13:29.732922+0000 0 0 2026-02-23T08:20:53.160 INFO:tasks.workunit.client.0.trial186.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:38.263019+0000 0'0 41:62 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T20:02:10.146438+0000 0 0 2026-02-23T08:20:53.160 INFO:tasks.workunit.client.0.trial186.stdout: 2026-02-23T08:20:53.160 INFO:tasks.workunit.client.0.trial186.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-23T08:20:53.160 INFO:tasks.workunit.client.0.trial186.stderr:dumped pgs 2026-02-23T08:20:53.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:20:53.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:20:53.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:20:53.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:20:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:53.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-23T08:20:53.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:20:53.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:20:53.197 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:20:53.197 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:20:53.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:20:53.408 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:20:53.408 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:20:53.408 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:20:53.408 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:20:53.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:20:53.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:20:53.718 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-23T08:20:53.925 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already down. 2026-02-23T08:20:53.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-23T08:20:54.179 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already out. 2026-02-23T08:20:54.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-23T08:20:54.405 INFO:tasks.workunit.client.0.trial186.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-23T08:20:54.405 INFO:tasks.workunit.client.0.trial186.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:47.296548+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-23T08:20:46.799275+0000 0'0 2026-02-23T08:20:46.799275+0000 0 0 periodic scrub scheduled @ 2026-02-24T18:31:15.062015+0000 0 0 2026-02-23T08:20:54.405 INFO:tasks.workunit.client.0.trial186.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.363223+0000 0'0 41:20 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T17:21:52.834667+0000 0 0 2026-02-23T08:20:54.405 INFO:tasks.workunit.client.0.trial186.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.334336+0000 0'0 41:94 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:35:02.623196+0000 0 0 2026-02-23T08:20:54.405 INFO:tasks.workunit.client.0.trial186.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.363715+0000 0'0 41:20 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T16:13:29.732922+0000 0 0 2026-02-23T08:20:54.406 INFO:tasks.workunit.client.0.trial186.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:38.263019+0000 0'0 41:62 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T20:02:10.146438+0000 0 0 2026-02-23T08:20:54.406 INFO:tasks.workunit.client.0.trial186.stdout: 2026-02-23T08:20:54.406 INFO:tasks.workunit.client.0.trial186.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-23T08:20:54.406 INFO:tasks.workunit.client.0.trial186.stderr:dumped pgs 2026-02-23T08:20:54.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-23T08:20:54.415 INFO:tasks.workunit.client.0.trial186.stderr:4+0 records in 2026-02-23T08:20:54.415 INFO:tasks.workunit.client.0.trial186.stderr:4+0 records out 2026-02-23T08:20:54.416 INFO:tasks.workunit.client.0.trial186.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.7155e-05 s, 61.0 MB/s 2026-02-23T08:20:54.416 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-23T08:20:54.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.868 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:54.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:54.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.070 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.070 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.251 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.395 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.737 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.737 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.957 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:55.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:55.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.423 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.639 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.765 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:20:56.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:20:56.841 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-23T08:20:56.842 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:20:56.842 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-23T08:20:56.842 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-23T08:20:56.842 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:20:57.060 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:2147483647' 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:20:57.061 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-23T08:20:57.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:20:57.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-23T08:20:57.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:20:57.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-23T08:20:57.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:20:57.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:20:57.063 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:20:57.064 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-23T08:20:57.064 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:20:57.064 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:20:57.064 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:57.064 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:57.064 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:57.064 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:20:57.065 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:20:57.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:20:57.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:20:57.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:20:57.131 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-23T08:20:57.131 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:20:57.131 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:20:57.131 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:57.131 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:57.132 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:57.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:20:57.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:20:57.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-23T08:20:57.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-23T08:20:57.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:20:57.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:20:57.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:20:57.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:20:57.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:20:57.195 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-23T08:20:57.196 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-23T08:20:57.196 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:20:57.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:20:57.414 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr:2147483647' 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:20:57.415 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-23T08:20:57.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:20:57.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-23T08:20:57.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:20:57.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:57.418 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:57.419 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:20:57.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:20:57.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:20:57.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:20:57.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:20:57.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-23T08:20:57.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:20:57.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:20:57.550 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:20:57.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:20:57.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:20:57.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:20:57.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:20:57.552 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:20:57.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:20:57.553 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:20:57.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:20:57.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:20:57.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:20:57.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:20:57.563 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:57.558+0000 7f09e119ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:57.564 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:57.558+0000 7f09e119ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:57.565 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:57.558+0000 7f09e119ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:20:57.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:20:57.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:20:57.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:20:57.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:20:57.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:20:57.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:57.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:20:57.768 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:20:57.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:57.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:20:57.940 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:57.934+0000 7f09e119ea40 -1 Falling back to public interface 2026-02-23T08:20:57.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:20:58.637 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:20:58.630+0000 7f09e119ea40 -1 osd.2 42 log_to_monitors true 2026-02-23T08:20:58.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:20:58.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:20:58.988 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:20:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:20:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:20:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:20:59.193 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up out weight 0 up_from 47 up_thru 19 down_at 42 last_clean_interval [18,42) [v2:127.0.0.1:6818/3742166403,v1:127.0.0.1:6819/3742166403] [v2:127.0.0.1:6820/3742166403,v1:127.0.0.1:6821/3742166403] exists,up 8cecc3c0-792f-4301-a978-dc84d3f824ae 2026-02-23T08:20:59.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:20:59.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:20:59.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:20:59.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-23T08:20:59.456 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already in. 2026-02-23T08:20:59.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-23T08:21:14.468 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-23T08:21:14.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-23T08:21:14.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-23T08:21:14.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-23T08:21:14.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-23T08:21:14.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-23T08:21:14.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-23T08:21:14.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-23T08:21:14.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-23T08:21:14.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-23T08:21:14.675 INFO:tasks.workunit.client.0.trial186.stdout:active+recovery_unfound+degraded 2026-02-23T08:21:14.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-23T08:21:14.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-23T08:21:14.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-23T08:21:14.880 INFO:tasks.workunit.client.0.trial186.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-23T08:21:14.880 INFO:tasks.workunit.client.0.trial186.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-23T08:20:59.504748+0000 45'101 52:360 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-23T08:20:46.799275+0000 0'0 2026-02-23T08:20:46.799275+0000 0 0 no scrub is scheduled 0 0 2026-02-23T08:21:14.880 INFO:tasks.workunit.client.0.trial186.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:59.501627+0000 0'0 52:59 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T13:14:14.853955+0000 0 0 2026-02-23T08:21:14.880 INFO:tasks.workunit.client.0.trial186.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.334336+0000 0'0 52:115 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:35:02.623196+0000 0 0 2026-02-23T08:21:14.881 INFO:tasks.workunit.client.0.trial186.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:45.363715+0000 0'0 52:41 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T16:13:29.732922+0000 0 0 2026-02-23T08:21:14.881 INFO:tasks.workunit.client.0.trial186.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-23T08:20:59.502747+0000 0'0 52:101 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-23T08:20:26.159665+0000 0'0 2026-02-23T08:20:26.159665+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:00:56.608302+0000 0 0 2026-02-23T08:21:14.881 INFO:tasks.workunit.client.0.trial186.stdout: 2026-02-23T08:21:14.881 INFO:tasks.workunit.client.0.trial186.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-23T08:21:14.881 INFO:tasks.workunit.client.0.trial186.stderr:dumped pgs 2026-02-23T08:21:14.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:21:14.979 INFO:tasks.workunit.client.0.trial186.stdout:{ 2026-02-23T08:21:14.979 INFO:tasks.workunit.client.0.trial186.stdout: "num_missing": 1, 2026-02-23T08:21:14.979 INFO:tasks.workunit.client.0.trial186.stdout: "num_unfound": 1, 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "objects": [ 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "oid": { 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "oid": "obj75", 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "key": "", 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "snapid": -2, 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "hash": 3565005477, 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "max": 0, 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "pool": 2, 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: "namespace": "" 2026-02-23T08:21:14.980 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "need": "45'76", 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "have": "0'0", 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "flags": "none", 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "locations": [ 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: ] 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "state": "NotRecovering", 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "available_might_have_unfound": true, 2026-02-23T08:21:14.981 INFO:tasks.workunit.client.0.trial186.stdout: "might_have_unfound": [], 2026-02-23T08:21:14.982 INFO:tasks.workunit.client.0.trial186.stdout: "more": false 2026-02-23T08:21:14.982 INFO:tasks.workunit.client.0.trial186.stdout:} 2026-02-23T08:21:14.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-23T08:21:15.063 INFO:tasks.workunit.client.0.trial186.stdout:{ 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: "snap_trimq": "[]", 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: "snap_trimq_len": 0, 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+recovery_unfound+degraded", 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: "epoch": 52, 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.064 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "acting_recovery_backfill": [ 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "2(4)", 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "info": { 2026-02-23T08:21:15.065 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s0", 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "0", 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "45'101", 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "45'75", 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "0'0", 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 101, 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "51", 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 39, 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 39, 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 50, 2026-02-23T08:21:15.066 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 39, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 50, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 50, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 39, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:21:15.067 INFO:tasks.workunit.client.0.trial186.stdout: "version": "45'101", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 361, 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 52, 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+recovery_unfound+degraded", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:20:59.660129+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:20:59.504748+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:20:59.660129+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:20:59.660129+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:20:53.189844+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:20:59.502726+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:20:59.502726+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:20:59.660129+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:20:59.500824+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:20:59.660129+0000", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 50, 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "0'0", 2026-02-23T08:21:15.068 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "0'0", 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "created": 39, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 0, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:21:15.069 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 413696, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 101, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:21:15.070 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 505, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 1, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 1, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 3, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 1, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 101, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 101, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 404, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:21:15.071 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:21:15.072 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.073 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.074 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 100 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),4(2)", 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 1 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:21:15.075 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "peer_info": [ 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "peer": "0(3)", 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s3", 2026-02-23T08:21:15.076 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "3", 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "45'101", 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "45'101", 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "0'0", 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 101, 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 39, 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 39, 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 50, 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 39, 2026-02-23T08:21:15.077 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 50, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 50, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 39, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "version": "45'101", 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 347, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 50, 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+remapped", 2026-02-23T08:21:15.078 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:20:53.189844+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:20:53.596885+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:20:53.596802+0000", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 50, 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "0'0", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "0'0", 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "created": 39, 2026-02-23T08:21:15.079 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 0, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:21:15.080 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-24T15:23:23.347570+0000", 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 413696, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 101, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 505, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 0, 2026-02-23T08:21:15.081 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 101, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 101, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 101, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 404, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:21:15.082 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:21:15.083 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.084 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:21:15.085 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 101 2026-02-23T08:21:15.086 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:21:15.087 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.088 INFO:tasks.workunit.client.0.trial186.stdout: "peer": "1(1)", 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s1", 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "1", 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "45'101", 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "45'101", 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "0'0", 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 101, 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:21:15.089 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 39, 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 39, 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 50, 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 39, 2026-02-23T08:21:15.090 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 50, 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 50, 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 39, 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.091 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: "version": "45'101", 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 131, 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 45, 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+remapped", 2026-02-23T08:21:15.092 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:20:53.189844+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:20:53.596885+0000", 2026-02-23T08:21:15.093 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:20:53.596802+0000", 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 50, 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "0'0", 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "0'0", 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "created": 39, 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.094 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 0, 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:21:15.095 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-24T15:23:23.347570+0000", 2026-02-23T08:21:15.096 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 413696, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 101, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 505, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 0, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 1, 2026-02-23T08:21:15.097 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 101, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 0, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 101, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 101, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 404, 2026-02-23T08:21:15.098 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:21:15.099 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:21:15.100 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.101 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:21:15.102 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 101 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:21:15.103 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:21:15.104 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "peer": "2(4)", 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s4", 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "4", 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "45'101", 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "45'75", 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "0'0", 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 101, 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:21:15.105 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 39, 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 39, 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 50, 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 39, 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:21:15.106 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 50, 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 50, 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 39, 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.107 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "version": "45'101", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 137, 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 49, 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+remapped", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:20:58.997264+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:20:58.895988+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:20:58.895988+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:20:58.791604+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:20:53.189844+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:20:58.997264+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:20:58.997264+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:20:58.997264+0000", 2026-02-23T08:21:15.108 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 50, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "0'0", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "0'0", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "created": 39, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 0, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:21:15.109 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 413696, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 101, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 505, 2026-02-23T08:21:15.110 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 1, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 101, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 101, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 404, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:21:15.111 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:21:15.112 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [], 2026-02-23T08:21:15.113 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [], 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "peer": "4(2)", 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s2", 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "shard": "2", 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "last_update": "45'101", 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "last_complete": "45'101", 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "log_tail": "0'0", 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "last_user_version": 101, 2026-02-23T08:21:15.114 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill": "MAX", 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete": [], 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [], 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "history": { 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_created": 39, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "epoch_pool_created": 39, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_started": 50, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_interval_clean": 39, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_split": 0, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_marked_full": 0, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "same_up_since": 50, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "same_interval_since": 50, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "same_primary_since": 39, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "prior_readable_until_ub": 0 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "stats": { 2026-02-23T08:21:15.115 INFO:tasks.workunit.client.0.trial186.stdout: "version": "45'101", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "reported_seq": 131, 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "reported_epoch": 45, 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "state": "active+remapped", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_fresh": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_change": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_active": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_peered": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean": "2026-02-23T08:20:53.189844+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_active": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_became_peered": "2026-02-23T08:20:53.598405+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_unstale": "2026-02-23T08:20:56.835616+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_undegraded": "2026-02-23T08:20:53.596885+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_fullsized": "2026-02-23T08:20:53.596802+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "mapping_epoch": 50, 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "log_start": "0'0", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_start": "0'0", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "created": 39, 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_clean": 40, 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "parent": "0.0", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "parent_split_bits": 0, 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub": "0'0", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub": "0'0", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_deep_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.116 INFO:tasks.workunit.client.0.trial186.stdout: "last_clean_scrub_stamp": "2026-02-23T08:20:46.799275+0000", 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "objects_scrubbed": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "log_size": 101, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "log_dups_size": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "ondisk_log_size": 101, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "stats_invalid": false, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "dirty_stats_invalid": false, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "omap_stats_invalid": false, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_stats_invalid": false, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "hitset_bytes_stats_invalid": false, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "pin_stats_invalid": false, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "manifest_stats_invalid": false, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrimq_len": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "last_scrub_duration": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-24T15:23:23.347570+0000", 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "scrub_duration": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "objects_trimmed": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "snaptrim_duration": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "stat_sum": { 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes": 413696, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects": 101, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_clones": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "num_object_copies": 505, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing_on_primary": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_missing": 0, 2026-02-23T08:21:15.117 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_degraded": 101, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_misplaced": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_unfound": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_dirty": 101, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_whiteouts": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_read": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_read_kb": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_write": 101, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_write_kb": 404, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_scrub_errors": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_shallow_scrub_errors": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_deep_scrub_errors": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_recovered": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_recovered": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_keys_recovered": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_omap": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_hit_set_archive": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_bytes_hit_set_archive": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_kb": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_kb": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_promote": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_high": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_flush_mode_low": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_some": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_evict_mode_full": 0, 2026-02-23T08:21:15.118 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_pinned": 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "num_legacy_snapsets": 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "num_large_omap_objects": 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_manifest": 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_bytes": 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "num_omap_keys": 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "num_objects_repaired": 0 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "up": [ 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "acting": [ 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 3, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 1, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 4, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 0, 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: 2 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "avail_no_missing": [ 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "3(0)", 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "0(3)", 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "1(1)", 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: "4(2)" 2026-02-23T08:21:15.119 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "object_location_counts": [ 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "objects": 101 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "blocked_by": [], 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "up_primary": 3, 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "acting_primary": 3, 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "purged_snaps": [] 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "empty": 0, 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "dne": 0, 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "incomplete": 0, 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "last_epoch_started": 51, 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "hit_set_history": { 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "current_last_update": "0'0", 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "history": [] 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "recovery_state": [ 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "name": "Started/Primary/Active", 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "enter_time": "2026-02-23T08:20:59.500639+0000", 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: "might_have_unfound": [ 2026-02-23T08:21:15.120 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "0(3)", 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "status": "already probed" 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "1(1)", 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "status": "already probed" 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "2(4)", 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "status": "already probed" 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "osd": "4(2)", 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "status": "already probed" 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "recovery_progress": { 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "backfill_targets": [], 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "waiting_on_backfill": [], 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "last_backfill_started": "MIN", 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "backfill_info": { 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "begin": "MIN", 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "end": "MIN", 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "objects": [] 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.121 INFO:tasks.workunit.client.0.trial186.stdout: "peer_backfill_info": [], 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "backfills_in_flight": [], 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "recovering": [], 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "pg_backend": { 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "recovery_ops": [], 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "read_ops": [] 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: { 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "name": "Started", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "enter_time": "2026-02-23T08:20:59.400568+0000" 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: ], 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "scrubber": { 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "active": false, 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "must_scrub": false, 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "must_deep_scrub": false, 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "must_repair": false, 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "sched_time": "2026-02-24T15:23:23.347570+0000", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "schedule": "not registered for scrubbing", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "urgency": "periodic-regular", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "shallow-target": { 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s0", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "level": "shallow", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "urgency": "periodic-regular", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "sched_time": "2026-02-24T15:23:23.347570+0000", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "orig_sched_time": "2026-02-24T15:23:23.347570+0000", 2026-02-23T08:21:15.122 INFO:tasks.workunit.client.0.trial186.stdout: "last_issue": "ok", 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "forced": false, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "eligible": false, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "queued": false, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "active": false 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "deep-target": { 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "pgid": "2.0s0", 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "level": "deep", 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "urgency": "periodic-regular", 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "sched_time": "2026-02-27T20:58:56.733801+0000", 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "orig_sched_time": "2026-02-27T20:58:56.733801+0000", 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "last_issue": "ok", 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "forced": false, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "eligible": false, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "queued": false, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "active": false 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: } 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: }, 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout: "agent_state": {} 2026-02-23T08:21:15.123 INFO:tasks.workunit.client.0.trial186.stdout:} 2026-02-23T08:21:15.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:21:15.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-23T08:21:15.156 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:21:15.156 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-23T08:21:15.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-23T08:21:15.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-23T08:21:15.235 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-23T08:21:15.235 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-23T08:21:15.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-23T08:21:15.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-23T08:21:15.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-23T08:21:20.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-23T08:21:20.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-23T08:21:20.388 INFO:tasks.workunit.client.0.trial186.stderr:pg has no unfound objects 2026-02-23T08:21:20.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-23T08:21:20.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:21:20.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:21:20.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:21:20.397 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:21:20.398 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:21:20.398 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:21:20.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:21:20.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:21:20.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:21:20.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:21:20.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:21:20.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:21:20.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:21:20.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:21:20.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:21:20.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:21:20.721 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:21:20.721 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:21:20.721 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:21:20.722 INFO:tasks.workunit.client.0.trial186.stderr:4' 2026-02-23T08:21:20.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:21:20.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:20.722 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:21:20.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-23T08:21:20.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-23T08:21:20.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-23T08:21:20.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:20.804 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:21:20.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574862 2026-02-23T08:21:20.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574862 2026-02-23T08:21:20.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574862' 2026-02-23T08:21:20.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:20.886 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:21:20.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462919 2026-02-23T08:21:20.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462919 2026-02-23T08:21:20.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574862 2-201863462919' 2026-02-23T08:21:20.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:20.966 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:21:21.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182412 2026-02-23T08:21:21.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182412 2026-02-23T08:21:21.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574862 2-201863462919 3-107374182412' 2026-02-23T08:21:21.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:21.048 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:21:21.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888076 2026-02-23T08:21:21.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888076 2026-02-23T08:21:21.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574862 2-201863462919 3-107374182412 4-146028888076' 2026-02-23T08:21:21.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:21.128 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-23T08:21:21.128 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:21.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:21:21.130 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-23T08:21:21.130 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:21.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-23T08:21:21.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-23T08:21:21.132 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803791 2026-02-23T08:21:21.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:21:21.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803791 2026-02-23T08:21:21.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:21:22.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:21:22.354 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:21:22.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-23T08:21:22.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:22.572 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574862 2026-02-23T08:21:22.572 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:22.573 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:21:22.574 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574862 2026-02-23T08:21:22.574 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:22.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574862 2026-02-23T08:21:22.576 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574862 2026-02-23T08:21:22.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574862' 2026-02-23T08:21:22.576 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:21:22.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574862 -lt 55834574862 2026-02-23T08:21:22.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:22.793 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-201863462919 2026-02-23T08:21:22.793 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:22.794 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:21:22.795 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-201863462919 2026-02-23T08:21:22.795 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:22.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=201863462919 2026-02-23T08:21:22.796 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 201863462919 2026-02-23T08:21:22.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 201863462919' 2026-02-23T08:21:22.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:21:23.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-23T08:21:23.011 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:23.012 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182412 2026-02-23T08:21:23.012 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:23.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:21:23.014 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182412 2026-02-23T08:21:23.014 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:23.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182412 2026-02-23T08:21:23.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182412' 2026-02-23T08:21:23.016 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182412 2026-02-23T08:21:23.016 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:21:23.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182413 -lt 107374182412 2026-02-23T08:21:23.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:23.230 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-146028888076 2026-02-23T08:21:23.230 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:23.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:21:23.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888076 2026-02-23T08:21:23.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:23.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=146028888076 2026-02-23T08:21:23.234 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 146028888076 2026-02-23T08:21:23.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 146028888076' 2026-02-23T08:21:23.234 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:21:23.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 146028888076 -lt 146028888076 2026-02-23T08:21:23.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:21:23.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:21:23.453 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:21:23.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:21:23.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:21:23.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:21:23.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:21:23.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:21:23.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:21:23.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:21:23.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:21:23.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:21:23.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:21:23.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:21:23.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:21:24.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:21:24.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:21:24.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:21:24.225 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-23T08:21:24.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-23T08:21:24.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-23T08:21:24.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-23T08:21:24.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-23T08:21:24.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-23T08:21:24.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-23T08:21:24.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-23T08:21:24.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-23T08:21:24.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-23T08:21:24.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-23T08:21:24.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.446 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-23T08:21:24.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-23T08:21:24.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-23T08:21:24.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-23T08:21:24.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.527 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-23T08:21:24.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-23T08:21:24.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.571 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.573 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-23T08:21:24.573 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-23T08:21:24.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.614 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-23T08:21:24.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-23T08:21:24.636 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-23T08:21:24.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.679 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-23T08:21:24.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-23T08:21:24.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-23T08:21:24.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-23T08:21:24.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-23T08:21:24.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-23T08:21:24.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-23T08:21:24.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-23T08:21:24.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-23T08:21:24.854 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-23T08:21:24.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-23T08:21:24.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-23T08:21:24.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.943 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-23T08:21:24.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.962 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-23T08:21:24.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-23T08:21:24.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:24.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:24.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-23T08:21:24.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-23T08:21:25.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-23T08:21:25.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-23T08:21:25.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-23T08:21:25.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-23T08:21:25.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-23T08:21:25.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-23T08:21:25.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-23T08:21:25.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-23T08:21:25.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-23T08:21:25.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-23T08:21:25.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-23T08:21:25.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.274 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-23T08:21:25.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-23T08:21:25.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.314 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-23T08:21:25.316 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-23T08:21:25.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-23T08:21:25.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.386 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.387 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-23T08:21:25.387 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-23T08:21:25.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-23T08:21:25.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-23T08:21:25.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-23T08:21:25.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-23T08:21:25.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-23T08:21:25.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-23T08:21:25.543 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-23T08:21:25.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-23T08:21:25.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-23T08:21:25.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-23T08:21:25.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-23T08:21:25.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-23T08:21:25.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.694 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.694 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-23T08:21:25.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-23T08:21:25.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.737 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-23T08:21:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.761 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-23T08:21:25.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-23T08:21:25.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-23T08:21:25.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-23T08:21:25.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.850 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-23T08:21:25.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.871 INFO:tasks.workunit.client.0.trial186.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-23T08:21:25.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-23T08:21:25.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-23T08:21:25.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-23T08:21:25.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-23T08:21:25.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-23T08:21:25.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-23T08:21:25.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:25.986 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:25.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-23T08:21:25.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-23T08:21:26.010 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.033 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-23T08:21:26.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-23T08:21:26.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-23T08:21:26.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-23T08:21:26.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-23T08:21:26.121 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-23T08:21:26.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.165 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-23T08:21:26.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-23T08:21:26.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.213 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-23T08:21:26.215 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-23T08:21:26.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-23T08:21:26.263 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.285 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-23T08:21:26.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-23T08:21:26.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-23T08:21:26.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-23T08:21:26.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-23T08:21:26.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-23T08:21:26.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-23T08:21:26.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-23T08:21:26.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-23T08:21:26.446 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-23T08:21:26.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:21:26.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:21:26.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:21:26.714 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:21:26.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:21:27.000 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:21:27.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:21:27.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:21:27.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:21:27.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:21:27.013 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:21:27.013 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:21:27.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:21:27.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:21:27.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:21:27.124 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:21:27.124 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:21:27.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:21:27.126 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:21:27.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:21:27.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:21:27.128 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:21:27.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:21:27.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:21:27.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:21:27.130 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:21:27.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:21:27.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:21:27.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:21:27.159 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:21:27.160 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:27.160 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:27.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:21:27.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:21:27.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:21:27.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:21:27.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:21:27.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:21:27.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:21:27.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:21:27.164 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:21:27.164 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:21:27.165 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:21:27.166 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:21:27.166 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:21:27.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:21:27.167 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:21:27.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:21:27.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:21:27.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:21:27.169 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:21:27.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:21:27.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:21:27.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:21:27.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:21:27.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:27.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:27.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:21:27.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:21:27.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:21:27.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:21:27.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:21:27.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:27.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:27.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:21:27.177 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:21:27.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:21:27.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:21:27.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:21:27.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:21:27.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:21:27.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:21:27.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:21:27.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:21:27.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:21:27.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:21:27.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:21:27.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:27.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:27.200 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:27.200 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:27.200 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:27.200 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:27.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:21:27.230 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:21:27.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:21:27.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:21:27.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:21:27.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:21:27.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:21:27.233 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:21:27.233 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:21:27.234 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:21:27.234 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:21:27.234 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:27.234 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:27.234 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:21:27.234 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:21:27.234 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:21:27.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:21:27.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:21:27.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:21:27.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:21:27.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:21:27.291 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:21:27.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:21:27.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:21:27.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:21:27.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:21:27.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:21:27.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:21:27.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:21:27.456 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:21:27.456 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:27.456 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:27.456 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:27.456 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:27.457 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:27.457 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:27.457 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:21:27.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:21:27.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:21:27.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:21:27.585 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:21:27.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:21:28.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:21:28.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:21:28.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:21:28.597 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:21:28.597 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:28.597 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:28.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:21:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:21:28.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:21:28.645 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:21:28.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:21:28.653 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:21:27.214+0000 7fec34ef9ec0 0 load: jerasure load: lrc 2026-02-23T08:21:28.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-23T08:21:28.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-23T08:21:28.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-23T08:21:28.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-23T08:21:28.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-23T08:21:28.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:21:28.663 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-23T08:21:28.663 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-23T08:21:28.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:21:28.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:21:28.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:21:28.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:21:28.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:21:28.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:21:28.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:21:28.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:21:28.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:21:28.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:21:28.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:21:28.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:21:28.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:21:28.665 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:21:28.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=66567935-2145-4e67-ab6b-dc9dde09b2b6 2026-02-23T08:21:28.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 66567935-2145-4e67-ab6b-dc9dde09b2b6' 2026-02-23T08:21:28.668 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 66567935-2145-4e67-ab6b-dc9dde09b2b6 2026-02-23T08:21:28.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:21:28.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAIDpxpy97MJxAAUieabE3h3mnq1VoC33nCMw== 2026-02-23T08:21:28.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAIDpxpy97MJxAAUieabE3h3mnq1VoC33nCMw=="}' 2026-02-23T08:21:28.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 66567935-2145-4e67-ab6b-dc9dde09b2b6 -i td/test-erasure-eio/0/new.json 2026-02-23T08:21:28.782 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:28.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:21:28.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAIDpxpy97MJxAAUieabE3h3mnq1VoC33nCMw== --osd-uuid 66567935-2145-4e67-ab6b-dc9dde09b2b6 2026-02-23T08:21:28.808 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:28.802+0000 7fbf1da8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:28.809 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:28.802+0000 7fbf1da8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:28.810 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:28.802+0000 7fbf1da8ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:28.811 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:28.802+0000 7fbf1da8ca40 -1 bdev(0x5613a19a4400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:21:28.811 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:28.802+0000 7fbf1da8ca40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:21:29.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:21:29.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:21:29.480 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:21:29.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:21:29.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:21:29.592 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:21:29.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:21:29.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:21:29.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:21:29.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:21:29.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:21:29.607 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:29.598+0000 7fccda590a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:29.608 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:29.602+0000 7fccda590a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:29.609 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:29.602+0000 7fccda590a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:21:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:21:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:21:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:21:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:21:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:29.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:21:29.744 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:29.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:29.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:21:29.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:30.057 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:30.050+0000 7fccda590a40 -1 Falling back to public interface 2026-02-23T08:21:30.618 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:30.610+0000 7fccda590a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:21:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:30.856 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:21:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:30.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:21:31.078 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:32.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:32.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:32.080 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:21:32.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:21:32.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:32.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:21:32.298 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1588095407,v1:127.0.0.1:6803/1588095407] [v2:127.0.0.1:6804/1588095407,v1:127.0.0.1:6805/1588095407] exists,up 66567935-2145-4e67-ab6b-dc9dde09b2b6 2026-02-23T08:21:32.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:21:32.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:21:32.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:21:32.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:21:32.304 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=62d2a2d3-4c0a-40d0-b9b3-8335a72af1f5 2026-02-23T08:21:32.304 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 62d2a2d3-4c0a-40d0-b9b3-8335a72af1f5 2026-02-23T08:21:32.304 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 62d2a2d3-4c0a-40d0-b9b3-8335a72af1f5' 2026-02-23T08:21:32.304 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:21:32.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAMDpxpzbl/EhAAxmTOTH2/Ze7oIfK/ebyXrw== 2026-02-23T08:21:32.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAMDpxpzbl/EhAAxmTOTH2/Ze7oIfK/ebyXrw=="}' 2026-02-23T08:21:32.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 62d2a2d3-4c0a-40d0-b9b3-8335a72af1f5 -i td/test-erasure-eio/1/new.json 2026-02-23T08:21:32.542 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:32.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:21:32.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMDpxpzbl/EhAAxmTOTH2/Ze7oIfK/ebyXrw== --osd-uuid 62d2a2d3-4c0a-40d0-b9b3-8335a72af1f5 2026-02-23T08:21:32.570 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:32.562+0000 7f2ed8113a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:32.571 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:32.566+0000 7f2ed8113a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:32.572 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:32.566+0000 7f2ed8113a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:32.573 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:32.566+0000 7f2ed8113a40 -1 bdev(0x56411b00e400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:21:32.573 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:32.566+0000 7f2ed8113a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:21:33.210 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:21:33.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:21:33.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:21:33.212 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:21:33.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:21:33.509 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:21:33.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:21:33.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:21:33.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:21:33.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:21:33.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:21:33.532 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:33.526+0000 7f5fcc989a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:33.534 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:33.526+0000 7f5fcc989a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:33.535 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:33.530+0000 7f5fcc989a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:33.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:21:33.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:21:33.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:21:33.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:21:33.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:21:33.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:33.727 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:33.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:21:33.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:33.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:21:33.949 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:33.942+0000 7f5fcc989a40 -1 Falling back to public interface 2026-02-23T08:21:33.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:34.500 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:34.494+0000 7f5fcc989a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:21:34.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:34.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:34.959 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:34.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:21:34.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:34.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:21:35.172 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2476143276,v1:127.0.0.1:6811/2476143276] [v2:127.0.0.1:6812/2476143276,v1:127.0.0.1:6813/2476143276] exists,up 62d2a2d3-4c0a-40d0-b9b3-8335a72af1f5 2026-02-23T08:21:35.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:21:35.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:21:35.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:21:35.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:21:35.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:35.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:21:35.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:21:35.179 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:21:35.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8a76809d-028d-4d2a-b8d0-2f9563d7d415 2026-02-23T08:21:35.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 8a76809d-028d-4d2a-b8d0-2f9563d7d415' 2026-02-23T08:21:35.180 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 8a76809d-028d-4d2a-b8d0-2f9563d7d415 2026-02-23T08:21:35.180 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:21:35.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAPDpxpj8IBCxAAPTf+oQAr9z3W/THbl9+eWA== 2026-02-23T08:21:35.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAPDpxpj8IBCxAAPTf+oQAr9z3W/THbl9+eWA=="}' 2026-02-23T08:21:35.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8a76809d-028d-4d2a-b8d0-2f9563d7d415 -i td/test-erasure-eio/2/new.json 2026-02-23T08:21:35.416 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:21:35.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:21:35.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAPDpxpj8IBCxAAPTf+oQAr9z3W/THbl9+eWA== --osd-uuid 8a76809d-028d-4d2a-b8d0-2f9563d7d415 2026-02-23T08:21:35.452 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:35.446+0000 7f222f084a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:35.454 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:35.446+0000 7f222f084a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:35.455 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:35.450+0000 7f222f084a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:35.456 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:35.450+0000 7f222f084a40 -1 bdev(0x55a1fbd6e400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:21:35.456 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:35.450+0000 7f222f084a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:21:36.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:21:36.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:21:36.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:21:36.112 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:21:36.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:21:36.409 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:21:36.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:21:36.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:21:36.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:21:36.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:21:36.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:21:36.426 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:36.417+0000 7f37c7732a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:36.427 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:36.417+0000 7f37c7732a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:36.428 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:36.421+0000 7f37c7732a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:36.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:21:36.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:21:36.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:21:36.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:21:36.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:21:36.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:36.621 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:36.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:21:36.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:36.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:21:36.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:36.877 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:36.869+0000 7f37c7732a40 -1 Falling back to public interface 2026-02-23T08:21:37.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:37.477+0000 7f37c7732a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:21:37.836 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:37.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:37.837 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:37.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:21:37.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:37.837 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:21:38.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:39.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:39.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:39.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:21:39.054 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:21:39.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:39.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:21:39.270 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/468542934,v1:127.0.0.1:6819/468542934] [v2:127.0.0.1:6820/468542934,v1:127.0.0.1:6821/468542934] exists,up 8a76809d-028d-4d2a-b8d0-2f9563d7d415 2026-02-23T08:21:39.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:21:39.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:21:39.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:21:39.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:21:39.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:21:39.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:21:39.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:39.272 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:21:39.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:21:39.274 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:21:39.274 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:21:39.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7e00ec43-71b3-4c22-a006-b25cf50d5cc6 2026-02-23T08:21:39.276 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 7e00ec43-71b3-4c22-a006-b25cf50d5cc6 2026-02-23T08:21:39.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 7e00ec43-71b3-4c22-a006-b25cf50d5cc6' 2026-02-23T08:21:39.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:21:39.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQATDpxpLezrEBAA8dP9u8uPPJFs9zaGNo3Xlg== 2026-02-23T08:21:39.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQATDpxpLezrEBAA8dP9u8uPPJFs9zaGNo3Xlg=="}' 2026-02-23T08:21:39.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7e00ec43-71b3-4c22-a006-b25cf50d5cc6 -i td/test-erasure-eio/3/new.json 2026-02-23T08:21:39.509 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:21:39.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:21:39.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQATDpxpLezrEBAA8dP9u8uPPJFs9zaGNo3Xlg== --osd-uuid 7e00ec43-71b3-4c22-a006-b25cf50d5cc6 2026-02-23T08:21:39.544 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:39.537+0000 7f509d063a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:39.546 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:39.537+0000 7f509d063a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:39.547 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:39.537+0000 7f509d063a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:39.547 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:39.541+0000 7f509d063a40 -1 bdev(0x555706998400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:21:39.547 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:39.541+0000 7f509d063a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:21:40.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:21:40.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:21:40.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:21:40.220 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:21:40.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:21:40.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:21:40.522 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:21:40.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:21:40.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:21:40.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:21:40.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:21:40.546 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:40.537+0000 7f64940bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:40.548 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:40.541+0000 7f64940bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:40.549 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:40.541+0000 7f64940bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:40.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:21:40.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:21:40.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:21:40.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:21:40.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:21:40.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:40.735 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:40.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:21:40.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:40.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:21:40.913 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:40.905+0000 7f64940bea40 -1 Falling back to public interface 2026-02-23T08:21:40.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:41.517 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:41.509+0000 7f64940bea40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:21:41.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:41.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:41.960 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:41.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:21:41.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:41.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:21:42.186 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/323736694,v1:127.0.0.1:6827/323736694] [v2:127.0.0.1:6828/323736694,v1:127.0.0.1:6829/323736694] exists,up 7e00ec43-71b3-4c22-a006-b25cf50d5cc6 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-23T08:21:42.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:21:42.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:21:42.189 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-23T08:21:42.190 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:21:42.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e120c5db-f97e-4e92-80ff-80fd4aa8586a 2026-02-23T08:21:42.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 e120c5db-f97e-4e92-80ff-80fd4aa8586a' 2026-02-23T08:21:42.191 INFO:tasks.workunit.client.0.trial186.stdout:add osd4 e120c5db-f97e-4e92-80ff-80fd4aa8586a 2026-02-23T08:21:42.191 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:21:42.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAWDpxp24vYCxAAtcAJ4Pvgt2F3MC+F4py91A== 2026-02-23T08:21:42.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAWDpxp24vYCxAAtcAJ4Pvgt2F3MC+F4py91A=="}' 2026-02-23T08:21:42.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e120c5db-f97e-4e92-80ff-80fd4aa8586a -i td/test-erasure-eio/4/new.json 2026-02-23T08:21:42.422 INFO:tasks.workunit.client.0.trial186.stdout:4 2026-02-23T08:21:42.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-23T08:21:42.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAWDpxp24vYCxAAtcAJ4Pvgt2F3MC+F4py91A== --osd-uuid e120c5db-f97e-4e92-80ff-80fd4aa8586a 2026-02-23T08:21:42.456 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:42.449+0000 7fc1009b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:42.457 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:42.449+0000 7fc1009b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:42.458 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:42.449+0000 7fc1009b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:42.459 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:42.449+0000 7fc1009b2a40 -1 bdev(0x556a7c88a400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-23T08:21:42.459 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:42.449+0000 7fc1009b2a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-23T08:21:43.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-23T08:21:43.063 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:21:43.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-23T08:21:43.064 INFO:tasks.workunit.client.0.trial186.stdout:adding osd4 key to auth repository 2026-02-23T08:21:43.064 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:21:43.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-23T08:21:43.356 INFO:tasks.workunit.client.0.trial186.stdout:start osd.4 2026-02-23T08:21:43.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:21:43.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:21:43.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:21:43.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:21:43.378 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:43.369+0000 7f408c205a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:43.380 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:43.373+0000 7f408c205a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:43.381 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:43.373+0000 7f408c205a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:21:43.569 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:43.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:43.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:21:43.721 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:43.713+0000 7f408c205a40 -1 Falling back to public interface 2026-02-23T08:21:43.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:44.317 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:44.309+0000 7f408c205a40 -1 osd.4 0 log_to_monitors true 2026-02-23T08:21:44.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:44.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:44.789 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:44.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:21:44.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:44.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-23T08:21:45.003 INFO:tasks.workunit.client.0.trial186.stdout:osd.4 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/163406361,v1:127.0.0.1:6835/163406361] [v2:127.0.0.1:6836/163406361,v1:127.0.0.1:6837/163406361] exists,up e120c5db-f97e-4e92-80ff-80fd4aa8586a 2026-02-23T08:21:45.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:21:45.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:21:45.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:21:45.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:21:45.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-23T08:21:45.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:45.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:21:45.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:21:45.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:21:45.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:21:45.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-23T08:21:45.007 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:21:45.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=273ebb7a-c16b-42f3-8476-92105ffba694 2026-02-23T08:21:45.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 273ebb7a-c16b-42f3-8476-92105ffba694' 2026-02-23T08:21:45.008 INFO:tasks.workunit.client.0.trial186.stdout:add osd5 273ebb7a-c16b-42f3-8476-92105ffba694 2026-02-23T08:21:45.008 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:21:45.021 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAZDpxpnej6ABAApRb7DS3yis3r+vSyP6O6KA== 2026-02-23T08:21:45.021 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAZDpxpnej6ABAApRb7DS3yis3r+vSyP6O6KA=="}' 2026-02-23T08:21:45.021 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 273ebb7a-c16b-42f3-8476-92105ffba694 -i td/test-erasure-eio/5/new.json 2026-02-23T08:21:45.247 INFO:tasks.workunit.client.0.trial186.stdout:5 2026-02-23T08:21:45.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-23T08:21:45.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAZDpxpnej6ABAApRb7DS3yis3r+vSyP6O6KA== --osd-uuid 273ebb7a-c16b-42f3-8476-92105ffba694 2026-02-23T08:21:45.274 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:45.265+0000 7f3067f13a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:45.276 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:45.269+0000 7f3067f13a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:45.277 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:45.269+0000 7f3067f13a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:45.278 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:45.269+0000 7f3067f13a40 -1 bdev(0x56542d92a400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-23T08:21:45.278 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:45.269+0000 7f3067f13a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-23T08:21:45.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-23T08:21:45.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:21:45.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-23T08:21:45.981 INFO:tasks.workunit.client.0.trial186.stdout:adding osd5 key to auth repository 2026-02-23T08:21:45.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:21:46.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-23T08:21:46.280 INFO:tasks.workunit.client.0.trial186.stdout:start osd.5 2026-02-23T08:21:46.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:21:46.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:21:46.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:21:46.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:21:46.302 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:46.293+0000 7f6ddc8d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:46.304 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:46.297+0000 7f6ddc8d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:46.305 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:46.297+0000 7f6ddc8d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:21:46.489 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:46.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:46.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:21:46.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:46.725 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:46.717+0000 7f6ddc8d8a40 -1 Falling back to public interface 2026-02-23T08:21:47.289 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:47.281+0000 7f6ddc8d8a40 -1 osd.5 0 log_to_monitors true 2026-02-23T08:21:47.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:47.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:47.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:21:47.714 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:47.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:47.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-23T08:21:47.934 INFO:tasks.workunit.client.0.trial186.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1048001643,v1:127.0.0.1:6843/1048001643] [v2:127.0.0.1:6844/1048001643,v1:127.0.0.1:6845/1048001643] exists,up 273ebb7a-c16b-42f3-8476-92105ffba694 2026-02-23T08:21:47.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:21:47.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:21:47.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:21:47.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:21:47.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-23T08:21:47.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:21:47.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:21:47.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:21:47.940 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:21:47.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:21:47.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:21:47.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:21:47.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:21:47.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:21:47.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:21:47.941 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-23T08:21:47.942 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:21:47.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b773f6a1-6524-4d2f-9fec-cdae3504d600 2026-02-23T08:21:47.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 b773f6a1-6524-4d2f-9fec-cdae3504d600' 2026-02-23T08:21:47.942 INFO:tasks.workunit.client.0.trial186.stdout:add osd6 b773f6a1-6524-4d2f-9fec-cdae3504d600 2026-02-23T08:21:47.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:21:47.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAbDpxpOh2OOBAAwvAI6y/WbmPZfaShmmj1oQ== 2026-02-23T08:21:47.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAbDpxpOh2OOBAAwvAI6y/WbmPZfaShmmj1oQ=="}' 2026-02-23T08:21:47.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b773f6a1-6524-4d2f-9fec-cdae3504d600 -i td/test-erasure-eio/6/new.json 2026-02-23T08:21:48.177 INFO:tasks.workunit.client.0.trial186.stdout:6 2026-02-23T08:21:48.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-23T08:21:48.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAbDpxpOh2OOBAAwvAI6y/WbmPZfaShmmj1oQ== --osd-uuid b773f6a1-6524-4d2f-9fec-cdae3504d600 2026-02-23T08:21:48.204 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:48.197+0000 7fbe02776a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:48.206 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:48.197+0000 7fbe02776a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:48.207 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:48.197+0000 7fbe02776a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:48.208 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:48.201+0000 7fbe02776a40 -1 bdev(0x55f780ca4400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-23T08:21:48.208 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:48.201+0000 7fbe02776a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-23T08:21:48.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-23T08:21:48.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:21:48.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-23T08:21:48.870 INFO:tasks.workunit.client.0.trial186.stdout:adding osd6 key to auth repository 2026-02-23T08:21:48.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:21:49.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-23T08:21:49.171 INFO:tasks.workunit.client.0.trial186.stdout:start osd.6 2026-02-23T08:21:49.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:21:49.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:21:49.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:21:49.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:21:49.186 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:49.177+0000 7efe09af5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:49.188 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:49.181+0000 7efe09af5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:49.189 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:49.181+0000 7efe09af5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:21:49.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-23T08:21:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:21:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-23T08:21:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:21:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:21:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:49.390 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:21:49.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:21:49.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:49.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:21:49.589 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:49.581+0000 7efe09af5a40 -1 Falling back to public interface 2026-02-23T08:21:49.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:50.252 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:21:50.245+0000 7efe09af5a40 -1 osd.6 0 log_to_monitors true 2026-02-23T08:21:50.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:50.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:50.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:21:50.608 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:21:50.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:50.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:21:50.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:21:51.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:21:51.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:21:51.840 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:21:51.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:21:51.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:21:51.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-23T08:21:52.056 INFO:tasks.workunit.client.0.trial186.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/555953656,v1:127.0.0.1:6851/555953656] [v2:127.0.0.1:6852/555953656,v1:127.0.0.1:6853/555953656] exists,up b773f6a1-6524-4d2f-9fec-cdae3504d600 2026-02-23T08:21:52.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:21:52.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:21:52.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:21:52.058 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:21:52.058 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:21:52.058 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:21:52.058 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:21:52.059 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:52.059 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:52.059 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:21:52.059 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:21:52.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:21:52.122 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:21:52.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:21:52.130 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:21:30.054+0000 7fccda590a40 0 load: jerasure load: lrc 2026-02-23T08:21:52.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-23T08:21:52.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-23T08:21:52.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:21:52.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:21:52.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-23T08:21:52.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:21:52.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-23T08:21:52.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:21:52.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-23T08:21:52.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:21:52.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:21:52.799 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:21:52.808 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:21:53.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:21:53.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:21:53.809 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:21:53.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:21:53.810 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:21:53.810 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:21:53.810 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:21:53.811 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:21:53.811 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:21:53.811 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:21:53.917 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:21:53.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:21:53.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:21:53.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:21:53.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:21:53.918 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:21:54.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:21:54.142 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:21:54.142 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:21:54.142 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:21:54.142 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:21:54.143 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:21:54.143 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:21:54.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:21:54.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:54.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:21:54.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-23T08:21:54.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-23T08:21:54.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783' 2026-02-23T08:21:54.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:54.220 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:21:54.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-23T08:21:54.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-23T08:21:54.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574854' 2026-02-23T08:21:54.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:54.302 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:21:54.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-23T08:21:54.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-23T08:21:54.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411334' 2026-02-23T08:21:54.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:54.380 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:21:54.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182405 2026-02-23T08:21:54.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182405 2026-02-23T08:21:54.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411334 3-107374182405' 2026-02-23T08:21:54.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:54.459 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:21:54.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920772 2026-02-23T08:21:54.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920772 2026-02-23T08:21:54.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411334 3-107374182405 4-141733920772' 2026-02-23T08:21:54.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:54.537 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:21:54.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626436 2026-02-23T08:21:54.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626436 2026-02-23T08:21:54.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411334 3-107374182405 4-141733920772 5-180388626436' 2026-02-23T08:21:54.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:21:54.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:21:54.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332099 2026-02-23T08:21:54.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332099 2026-02-23T08:21:54.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411334 3-107374182405 4-141733920772 5-180388626436 6-219043332099' 2026-02-23T08:21:54.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:54.697 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803783 2026-02-23T08:21:54.698 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:54.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:21:54.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-23T08:21:54.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:54.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803783 2026-02-23T08:21:54.700 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803783 2026-02-23T08:21:54.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-23T08:21:54.701 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:21:54.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-23T08:21:54.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:21:55.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:21:55.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:21:56.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-23T08:21:56.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:56.141 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574854 2026-02-23T08:21:56.141 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:56.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:21:56.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-23T08:21:56.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:56.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-23T08:21:56.145 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574854 2026-02-23T08:21:56.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-23T08:21:56.145 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:21:56.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574855 -lt 55834574854 2026-02-23T08:21:56.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:56.366 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411334 2026-02-23T08:21:56.366 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:56.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:21:56.368 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411334 2026-02-23T08:21:56.368 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:56.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-23T08:21:56.370 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411334 2026-02-23T08:21:56.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-23T08:21:56.371 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:21:56.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-23T08:21:56.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:56.593 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182405 2026-02-23T08:21:56.593 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:56.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:21:56.595 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182405 2026-02-23T08:21:56.595 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:56.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182405 2026-02-23T08:21:56.597 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182405 2026-02-23T08:21:56.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-23T08:21:56.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:21:56.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-23T08:21:56.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:56.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920772 2026-02-23T08:21:56.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:56.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:21:56.813 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920772 2026-02-23T08:21:56.814 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:56.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920772 2026-02-23T08:21:56.815 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 141733920772 2026-02-23T08:21:56.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920772' 2026-02-23T08:21:56.816 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:21:57.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920773 -lt 141733920772 2026-02-23T08:21:57.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:57.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626436 2026-02-23T08:21:57.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:57.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:21:57.043 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626436 2026-02-23T08:21:57.043 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:57.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626436 2026-02-23T08:21:57.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626436' 2026-02-23T08:21:57.044 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 180388626436 2026-02-23T08:21:57.045 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:21:57.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626436 -lt 180388626436 2026-02-23T08:21:57.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:21:57.270 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332099 2026-02-23T08:21:57.270 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:21:57.271 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-23T08:21:57.271 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332099 2026-02-23T08:21:57.272 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:21:57.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332099 2026-02-23T08:21:57.273 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.6 seq 219043332099 2026-02-23T08:21:57.274 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332099' 2026-02-23T08:21:57.274 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-23T08:21:57.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332099 -lt 219043332099 2026-02-23T08:21:57.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:21:57.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:21:57.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:21:57.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:21:57.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:21:57.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:21:57.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:21:57.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:21:57.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:21:57.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:21:57.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:21:58.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:21:58.010 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:21:58.010 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:21:58.010 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:21:58.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:21:58.319 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-23T08:21:58.320 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:21:58.320 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:21:58.320 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:21:58.320 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:21:58.543 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:21:58.543 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:21:58.544 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-23T08:21:58.546 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-23T08:21:58.547 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:21:58.548 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:21:58.548 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:21:58.548 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:58.548 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:58.548 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:21:58.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:21:58.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:21:58.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:21:58.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:21:58.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:21:58.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-23T08:21:58.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-23T08:21:58.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:21:58.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-23T08:21:58.668 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-23T08:21:58.668 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:21:58.890 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-23T08:21:58.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-23T08:21:58.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:21:58.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:21:58.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:21:58.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:21:58.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:21:59.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:21:59.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-23T08:21:59.417 INFO:tasks.workunit.client.0.trial186.stderr:osd.6 is already down. 2026-02-23T08:21:59.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-23T08:21:59.672 INFO:tasks.workunit.client.0.trial186.stderr:osd.6 is already out. 2026-02-23T08:21:59.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-23T08:21:59.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:21:59.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:21:59.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:21:59.683 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:21:59.684 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:21:59.684 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:21:59.685 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:21:59.685 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:21:59.685 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:21:59.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:21:59.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:21:59.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:21:59.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:21:59.793 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:21:59.793 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:3 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:4 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:5 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:6' 2026-02-23T08:22:00.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:22:00.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:00.009 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:22:00.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-23T08:22:00.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-23T08:22:00.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786' 2026-02-23T08:22:00.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:00.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:22:00.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-23T08:22:00.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-23T08:22:00.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857' 2026-02-23T08:22:00.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:00.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:22:00.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-23T08:22:00.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-23T08:22:00.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411337' 2026-02-23T08:22:00.253 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:00.253 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:22:00.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182408 2026-02-23T08:22:00.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182408 2026-02-23T08:22:00.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411337 3-107374182408' 2026-02-23T08:22:00.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:00.331 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-23T08:22:00.412 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920775 2026-02-23T08:22:00.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920775 2026-02-23T08:22:00.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411337 3-107374182408 4-141733920775' 2026-02-23T08:22:00.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:00.413 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-23T08:22:00.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626439 2026-02-23T08:22:00.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626439 2026-02-23T08:22:00.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411337 3-107374182408 4-141733920775 5-180388626439' 2026-02-23T08:22:00.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:00.493 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-23T08:22:00.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-23T08:22:00.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-23T08:22:00.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-23T08:22:00.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-23T08:22:00.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:00.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803786 2026-02-23T08:22:00.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:00.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:22:00.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-23T08:22:00.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:00.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803786 2026-02-23T08:22:00.558 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803786 2026-02-23T08:22:00.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-23T08:22:00.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:00.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-23T08:22:00.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:22:01.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:22:01.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:01.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-23T08:22:02.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:02.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574857 2026-02-23T08:22:02.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:02.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:22:02.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-23T08:22:02.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:02.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-23T08:22:02.004 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574857 2026-02-23T08:22:02.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-23T08:22:02.005 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:22:02.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-23T08:22:02.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:02.230 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411337 2026-02-23T08:22:02.230 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:02.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:22:02.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411337 2026-02-23T08:22:02.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:02.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-23T08:22:02.233 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411337 2026-02-23T08:22:02.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-23T08:22:02.233 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:22:02.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-23T08:22:02.461 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:02.462 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182408 2026-02-23T08:22:02.462 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:02.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:22:02.464 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182408 2026-02-23T08:22:02.464 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:02.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182408 2026-02-23T08:22:02.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182408' 2026-02-23T08:22:02.466 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182408 2026-02-23T08:22:02.466 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:02.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182408 -lt 107374182408 2026-02-23T08:22:02.694 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:02.694 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920775 2026-02-23T08:22:02.695 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:02.695 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-23T08:22:02.696 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920775 2026-02-23T08:22:02.697 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:02.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920775 2026-02-23T08:22:02.698 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.4 seq 141733920775 2026-02-23T08:22:02.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920775' 2026-02-23T08:22:02.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-23T08:22:02.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920776 -lt 141733920775 2026-02-23T08:22:02.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:02.917 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626439 2026-02-23T08:22:02.917 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:02.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-23T08:22:02.918 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626439 2026-02-23T08:22:02.919 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:02.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626439 2026-02-23T08:22:02.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626439' 2026-02-23T08:22:02.921 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.5 seq 180388626439 2026-02-23T08:22:02.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-23T08:22:03.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626439 -lt 180388626439 2026-02-23T08:22:03.142 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:22:03.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:03.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:03.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:22:03.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:22:03.434 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:22:03.434 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:22:03.434 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:22:03.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:22:03.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:22:03.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:22:03.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:22:03.653 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:22:03.654 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:03.654 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:03.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:22:03.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:22:03.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:22:03.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-23T08:22:03.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:22:03.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:22:03.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-23T08:22:03.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:22:03.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:22:03.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-23T08:22:03.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:22:03.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:22:03.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:22:03.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:22:03.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:22:04.231 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:22:04.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:22:04.514 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:22:04.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:22:04.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:22:04.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:22:04.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:22:04.525 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:04.525 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:04.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:04.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:04.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:04.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:04.838 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:22:04.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:22:04.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:22:04.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:22:04.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:22:04.842 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:22:04.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:22:04.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:22:04.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:22:04.844 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:22:04.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:22:04.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:22:04.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:22:04.866 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:22:04.866 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:04.867 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:04.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:22:04.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:22:04.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:22:04.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:22:04.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:22:04.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:04.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:04.871 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:04.872 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:22:04.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:22:04.873 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:22:04.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:22:04.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:22:04.875 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:22:04.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:22:04.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:22:04.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:22:04.877 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:22:04.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:22:04.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:22:04.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:22:04.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:22:04.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:04.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:04.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:22:04.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:22:04.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:22:04.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:22:04.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:22:04.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:04.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:04.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:22:04.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:22:04.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:22:04.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:22:04.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:22:04.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:22:04.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:22:04.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:22:04.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:22:04.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:22:04.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:22:04.887 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:22:04.908 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:22:04.908 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:04.908 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:04.909 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:04.909 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:04.909 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:04.909 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:04.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:22:04.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:22:04.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:22:04.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:22:04.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:22:04.940 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:22:04.941 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:22:04.942 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:22:04.942 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:22:04.942 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:22:04.942 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:22:04.942 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:04.942 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:04.942 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:22:04.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:22:04.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:22:04.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:22:04.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:22:04.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:22:04.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:22:05.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:22:05.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:22:05.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:22:05.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:22:05.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:22:05.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:22:05.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:22:05.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:22:05.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:22:05.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:22:05.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:22:05.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:05.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:05.173 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:05.173 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:05.173 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:05.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:05.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:22:05.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:22:05.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:22:05.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:22:05.300 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:22:05.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:22:06.311 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:22:06.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:22:06.362 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:22:06.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:22:06.369 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:22:04.921+0000 7f616e0f0ec0 0 load: jerasure load: lrc 2026-02-23T08:22:06.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-23T08:22:06.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-23T08:22:06.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-23T08:22:06.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-23T08:22:06.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:22:06.371 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-23T08:22:06.372 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-23T08:22:06.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:22:06.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:22:06.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:06.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:22:06.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:06.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:22:06.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:22:06.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:22:06.380 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:22:06.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=49f31ecd-a891-4966-b45b-dd1fbd903680 2026-02-23T08:22:06.381 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 49f31ecd-a891-4966-b45b-dd1fbd903680 2026-02-23T08:22:06.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 49f31ecd-a891-4966-b45b-dd1fbd903680' 2026-02-23T08:22:06.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:22:06.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAuDpxpWJjoFhAAnh22AwxhK9LkmZk3SJYVHA== 2026-02-23T08:22:06.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAuDpxpWJjoFhAAnh22AwxhK9LkmZk3SJYVHA=="}' 2026-02-23T08:22:06.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 49f31ecd-a891-4966-b45b-dd1fbd903680 -i td/test-erasure-eio/0/new.json 2026-02-23T08:22:06.506 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:22:06.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:22:06.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAuDpxpWJjoFhAAnh22AwxhK9LkmZk3SJYVHA== --osd-uuid 49f31ecd-a891-4966-b45b-dd1fbd903680 2026-02-23T08:22:06.531 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:06.521+0000 7f1473779a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:06.532 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:06.525+0000 7f1473779a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:06.533 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:06.525+0000 7f1473779a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:06.533 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:06.525+0000 7f1473779a40 -1 bdev(0x556de488e400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:22:06.534 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:06.525+0000 7f1473779a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:22:07.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:22:07.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:22:07.242 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:22:07.243 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:22:07.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:22:07.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:22:07.393 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:22:07.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:07.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:22:07.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:22:07.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:22:07.407 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:07.397+0000 7f619faefa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:07.409 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:07.401+0000 7f619faefa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:07.410 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:07.401+0000 7f619faefa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:07.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:22:07.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:07.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:22:07.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:07.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:07.504 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:22:07.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:07.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:07.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:07.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:22:07.709 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:07.825 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:07.817+0000 7f619faefa40 -1 Falling back to public interface 2026-02-23T08:22:08.349 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:08.341+0000 7f619faefa40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:22:08.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:08.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:08.712 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:22:08.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:08.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:08.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:22:08.920 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:09.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:09.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:09.922 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:22:09.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:22:09.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:09.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:22:10.130 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3491814365,v1:127.0.0.1:6803/3491814365] [v2:127.0.0.1:6804/3491814365,v1:127.0.0.1:6805/3491814365] exists,up 49f31ecd-a891-4966-b45b-dd1fbd903680 2026-02-23T08:22:10.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:10.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:10.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:10.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:10.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:10.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:22:10.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:22:10.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:22:10.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:22:10.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:22:10.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d55d371e-5310-4a90-af6f-cbd9544a3f63' 2026-02-23T08:22:10.139 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:22:10.139 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:22:10.147 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAyDpxpAXt/CBAAo5IhwxZs/4lrB8yPzvug1g== 2026-02-23T08:22:10.147 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAyDpxpAXt/CBAAo5IhwxZs/4lrB8yPzvug1g=="}' 2026-02-23T08:22:10.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d55d371e-5310-4a90-af6f-cbd9544a3f63 -i td/test-erasure-eio/1/new.json 2026-02-23T08:22:10.371 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:22:10.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:22:10.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAyDpxpAXt/CBAAo5IhwxZs/4lrB8yPzvug1g== --osd-uuid d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:22:10.406 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:10.397+0000 7f14619aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:10.408 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:10.401+0000 7f14619aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:10.409 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:10.401+0000 7f14619aea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:10.410 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:10.401+0000 7f14619aea40 -1 bdev(0x5590a0918400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:22:10.410 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:10.401+0000 7f14619aea40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:22:11.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:22:11.027 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:22:11.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:22:11.028 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:22:11.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:22:11.328 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:22:11.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:22:11.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:11.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:22:11.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:22:11.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:22:11.352 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:11.341+0000 7f702ac6aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:11.353 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:11.345+0000 7f702ac6aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:11.355 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:11.345+0000 7f702ac6aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:11.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:22:11.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:11.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:22:11.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:11.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:11.551 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:22:11.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:11.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:11.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:11.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:22:11.690 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:11.681+0000 7f702ac6aa40 -1 Falling back to public interface 2026-02-23T08:22:11.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:12.285 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:12.277+0000 7f702ac6aa40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:22:12.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:12.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:12.767 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:22:12.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:12.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:12.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:22:12.975 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3638242008,v1:127.0.0.1:6811/3638242008] [v2:127.0.0.1:6812/3638242008,v1:127.0.0.1:6813/3638242008] exists,up d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:12.977 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:12.978 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:12.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:22:12.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:22:12.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:22:12.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:22:12.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=47151dc2-4e4e-48d7-9b36-3d8f90b761a2 2026-02-23T08:22:12.980 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 47151dc2-4e4e-48d7-9b36-3d8f90b761a2' 2026-02-23T08:22:12.980 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 47151dc2-4e4e-48d7-9b36-3d8f90b761a2 2026-02-23T08:22:12.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:22:12.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA0DpxpBTHQOhAAcCppaZNvm9xq7xa7HbHn2Q== 2026-02-23T08:22:12.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA0DpxpBTHQOhAAcCppaZNvm9xq7xa7HbHn2Q=="}' 2026-02-23T08:22:12.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 47151dc2-4e4e-48d7-9b36-3d8f90b761a2 -i td/test-erasure-eio/2/new.json 2026-02-23T08:22:13.213 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:22:13.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:22:13.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA0DpxpBTHQOhAAcCppaZNvm9xq7xa7HbHn2Q== --osd-uuid 47151dc2-4e4e-48d7-9b36-3d8f90b761a2 2026-02-23T08:22:13.239 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:13.229+0000 7fc8191b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:13.241 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:13.233+0000 7fc8191b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:13.242 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:13.233+0000 7fc8191b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:13.242 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:13.233+0000 7fc8191b3a40 -1 bdev(0x55ea9351c400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:22:13.243 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:13.233+0000 7fc8191b3a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:22:13.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:22:13.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:22:13.882 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:22:13.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:22:13.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:22:14.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:22:14.185 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:22:14.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:14.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:22:14.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:22:14.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:22:14.201 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:14.193+0000 7fc71c8daa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:14.203 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:14.193+0000 7fc71c8daa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:14.204 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:14.197+0000 7fc71c8daa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:22:14.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:14.621 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:14.613+0000 7fc71c8daa40 -1 Falling back to public interface 2026-02-23T08:22:15.236 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:15.229+0000 7fc71c8daa40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:22:15.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:15.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:15.621 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:22:15.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:15.621 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:15.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:22:15.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:16.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:16.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:16.843 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:22:16.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:22:16.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:16.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:22:17.059 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/102704994,v1:127.0.0.1:6819/102704994] [v2:127.0.0.1:6820/102704994,v1:127.0.0.1:6821/102704994] exists,up 47151dc2-4e4e-48d7-9b36-3d8f90b761a2 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:22:17.060 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:17.061 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:22:17.061 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:17.061 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:17.061 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:17.061 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:17.061 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:17.061 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:22:17.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:22:17.063 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:22:17.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:22:17.065 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 0edbfb58-b060-4ce0-b7a2-481902e6f5c3' 2026-02-23T08:22:17.065 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:22:17.065 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:22:17.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA5DpxpM8dTBBAAUieOV3yrj/PkYnqKo5LzUQ== 2026-02-23T08:22:17.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA5DpxpM8dTBBAAUieOV3yrj/PkYnqKo5LzUQ=="}' 2026-02-23T08:22:17.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 -i td/test-erasure-eio/3/new.json 2026-02-23T08:22:17.297 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:22:17.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:22:17.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5DpxpM8dTBBAAUieOV3yrj/PkYnqKo5LzUQ== --osd-uuid 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:22:17.322 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:17.313+0000 7fb9a3709a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:17.323 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:17.317+0000 7fb9a3709a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:17.324 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:17.317+0000 7fb9a3709a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:17.325 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:17.317+0000 7fb9a3709a40 -1 bdev(0x55cd6abc8400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:22:17.325 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:17.317+0000 7fb9a3709a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:22:18.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:22:18.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:22:18.015 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:22:18.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:22:18.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:22:18.311 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:22:18.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:22:18.312 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:18.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:22:18.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:22:18.313 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:22:18.336 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:18.325+0000 7fa17744fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:18.337 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:18.329+0000 7fa17744fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:18.339 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:18.329+0000 7fa17744fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:18.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:22:18.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:18.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:22:18.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:18.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:18.530 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:18.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:18.531 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:22:18.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:18.532 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:18.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:18.749 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:18.741+0000 7fa17744fa40 -1 Falling back to public interface 2026-02-23T08:22:19.319 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:19.309+0000 7fa17744fa40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:22:19.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:19.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:19.743 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:22:19.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:19.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:19.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:19.971 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3222616752,v1:127.0.0.1:6827/3222616752] [v2:127.0.0.1:6828/3222616752,v1:127.0.0.1:6829/3222616752] exists,up 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:22:19.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:19.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:19.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:19.973 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:22:19.973 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:22:19.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:22:19.974 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:22:19.974 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:19.974 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:19.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:22:19.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:22:19.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:22:20.036 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:22:20.042 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:22:20.045 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:22:07.821+0000 7f619faefa40 0 load: jerasure load: lrc 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:22:20.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-23T08:22:20.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:22:20.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:22:20.675 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:22:20.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:22:21.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:22:21.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:22:21.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:22:21.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:22:21.687 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:22:21.688 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:22:21.688 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:22:21.688 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:22:21.688 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:22:21.689 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:22:21.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:22:21.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:22:21.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:22:21.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:22:21.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:22:21.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:22:22.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:22:22.019 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:22.019 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:22:22.019 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:22:22.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:22:22.019 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:22.019 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:22:22.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-23T08:22:22.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-23T08:22:22.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-23T08:22:22.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:22.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:22:22.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-23T08:22:22.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-23T08:22:22.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-23T08:22:22.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:22.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:22:22.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444036 2026-02-23T08:22:22.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444036 2026-02-23T08:22:22.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444036' 2026-02-23T08:22:22.262 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:22.262 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:22:22.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182404 2026-02-23T08:22:22.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182404 2026-02-23T08:22:22.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444036 3-107374182404' 2026-02-23T08:22:22.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:22.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-23T08:22:22.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:22.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:22:22.346 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-23T08:22:22.346 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:22.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-23T08:22:22.348 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803781 2026-02-23T08:22:22.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-23T08:22:22.349 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:22.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-23T08:22:22.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:22:23.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:22:23.562 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:23.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-23T08:22:23.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:23.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-23T08:22:23.784 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:23.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:22:23.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-23T08:22:23.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:23.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-23T08:22:23.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-23T08:22:23.787 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607556 2026-02-23T08:22:23.788 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:22:24.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-23T08:22:24.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:24.013 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444036 2026-02-23T08:22:24.013 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:24.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:22:24.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444036 2026-02-23T08:22:24.015 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:24.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444036 2026-02-23T08:22:24.017 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-23T08:22:24.017 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444036 2026-02-23T08:22:24.017 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:22:24.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-23T08:22:24.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:24.238 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182404 2026-02-23T08:22:24.238 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:24.239 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:22:24.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182404 2026-02-23T08:22:24.240 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:24.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182404 2026-02-23T08:22:24.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182404' 2026-02-23T08:22:24.242 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182404 2026-02-23T08:22:24.242 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:24.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-23T08:22:24.463 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:22:24.464 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:24.464 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:24.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:22:24.740 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:22:24.740 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:22:24.740 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:22:24.740 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:22:24.740 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:22:24.741 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:22:24.741 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:22:24.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:22:24.953 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:22:24.953 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:24.953 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:25.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:22:25.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:22:25.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74343-0-10 2026-02-23T08:22:25.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74343-0-10 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74343-0-10 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:22:25.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74343-0-10 td/test-erasure-eio/ORIGINAL 2026-02-23T08:22:25.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74343-0-10 td/test-erasure-eio 0 10 set 2026-02-23T08:22:25.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-0-10 2026-02-23T08:22:25.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:22:25.264 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-0-10 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-0-10 2026-02-23T08:22:25.265 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-0-10 2026-02-23T08:22:25.266 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:22:25.489 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:22:25.489 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:25.489 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:22:25.489 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:22:25.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:22:25.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:22:25.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-23T08:22:25.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:22:25.780 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:22:25.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:22:25.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-23T08:22:25.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:10 bytes copied, 3.256e-05 s, 307 kB/s 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-23T08:22:25.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:25.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:26.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:26.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:26.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:22:26.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:22:26.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-23T08:22:26.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:22:26.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:26.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:26.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-23T08:22:26.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:22:26.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:26.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:22:26.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:22:26.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-23T08:22:26.420 INFO:tasks.workunit.client.0.trial186.stderr:start osd.3 2026-02-23T08:22:26.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:26.420 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-23T08:22:26.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-23T08:22:26.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:22:26.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:22:26.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:22:26.435 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:26.425+0000 7f77b0f35a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:26.436 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:26.429+0000 7f77b0f35a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:26.437 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:26.429+0000 7f77b0f35a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:26.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-23T08:22:26.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:26.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:22:26.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:26.631 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:26.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:26.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:26.632 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:22:26.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:26.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:26.782 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:26.773+0000 7f77b0f35a40 -1 Falling back to public interface 2026-02-23T08:22:26.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:27.382 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:27.373+0000 7f77b0f35a40 -1 osd.3 34 log_to_monitors true 2026-02-23T08:22:27.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:27.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:27.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:27.848 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:27.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:27.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:28.061 INFO:tasks.workunit.client.0.trial186.stderr:osd.3 up in weight 1 up_from 38 up_thru 38 down_at 35 last_clean_interval [25,34) [v2:127.0.0.1:6826/2971938740,v1:127.0.0.1:6827/2971938740] [v2:127.0.0.1:6828/2971938740,v1:127.0.0.1:6829/2971938740] exists,up 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:22:28.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:22:28.063 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:22:28.063 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:22:28.063 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:22:28.063 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:22:28.063 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:22:28.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:22:28.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:22:28.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:22:28.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:22:28.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:22:28.167 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:22:28.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:22:28.390 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:28.390 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:22:28.390 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:22:28.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:22:28.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:28.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:22:28.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-23T08:22:28.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-23T08:22:28.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-23T08:22:28.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:28.473 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:22:28.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607560 2026-02-23T08:22:28.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607560 2026-02-23T08:22:28.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560' 2026-02-23T08:22:28.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:28.556 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:22:28.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444039 2026-02-23T08:22:28.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444039 2026-02-23T08:22:28.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039' 2026-02-23T08:22:28.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:28.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:22:28.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757251 2026-02-23T08:22:28.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757251 2026-02-23T08:22:28.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-73014444039 3-163208757251' 2026-02-23T08:22:28.713 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:28.714 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-23T08:22:28.714 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:28.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:22:28.716 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-23T08:22:28.716 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:28.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-23T08:22:28.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-23T08:22:28.717 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803784 2026-02-23T08:22:28.717 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:28.937 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-23T08:22:28.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:22:29.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:22:29.939 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:30.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-23T08:22:30.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:30.162 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607560 2026-02-23T08:22:30.162 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:30.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:22:30.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607560 2026-02-23T08:22:30.164 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:30.165 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607560 2026-02-23T08:22:30.165 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607560' 2026-02-23T08:22:30.165 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 51539607560 2026-02-23T08:22:30.166 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:22:30.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607560 -lt 51539607560 2026-02-23T08:22:30.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:30.382 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444039 2026-02-23T08:22:30.383 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:30.383 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:22:30.384 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444039 2026-02-23T08:22:30.385 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:30.386 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444039 2026-02-23T08:22:30.386 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444039' 2026-02-23T08:22:30.386 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444039 2026-02-23T08:22:30.387 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:22:30.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-23T08:22:30.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:30.610 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-163208757251 2026-02-23T08:22:30.610 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:30.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:22:30.612 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-163208757251 2026-02-23T08:22:30.612 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:30.613 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757251 2026-02-23T08:22:30.613 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 163208757251' 2026-02-23T08:22:30.613 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 163208757251 2026-02-23T08:22:30.614 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:30.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-23T08:22:30.841 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:22:30.842 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:30.842 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:31.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:22:31.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:22:31.135 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:22:31.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:22:31.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:22:31.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:22:31.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:22:31.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:22:31.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:22:31.352 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:22:31.352 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:31.352 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:31.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:22:31.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:22:31.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:22:31.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:22:31.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:22:31.959 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:22:31.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-0-10 2026-02-23T08:22:31.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:22:31.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:22:31.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-0-10 2026-02-23T08:22:31.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:22:31.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:22:31.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-0-10 td/test-erasure-eio/COPY 2026-02-23T08:22:31.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:22:31.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:22:31.997 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74343-0-10 td/test-erasure-eio 1 10 set 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-0-10 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:22:32.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-23T08:22:32.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:22:32.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:22:32.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:22:32.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-0-10 2026-02-23T08:22:32.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:22:32.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-0-10 2026-02-23T08:22:32.001 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-0-10 2026-02-23T08:22:32.002 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:22:32.216 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:22:32.216 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:32.216 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:22:32.216 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:22:32.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:22:32.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:22:32.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-23T08:22:32.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:22:32.525 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:22:32.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:22:32.536 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-23T08:22:32.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:10 bytes copied, 4.9653e-05 s, 201 kB/s 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:32.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:32.540 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:32.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:32.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:32.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:22:32.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:22:32.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:22:32.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:22:32.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:22:32.849 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:33.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:33.138 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:22:33.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:22:33.140 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:22:33.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:33.140 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:22:33.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:22:33.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:22:33.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:22:33.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:22:33.157 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:33.149+0000 7fee26bc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:33.158 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:33.149+0000 7fee26bc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:33.159 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:33.149+0000 7fee26bc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:33.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:22:33.538 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:33.529+0000 7fee26bc3a40 -1 Falling back to public interface 2026-02-23T08:22:33.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:34.177 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:34.168+0000 7fee26bc3a40 -1 osd.1 44 log_to_monitors true 2026-02-23T08:22:34.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:34.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:34.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:34.575 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:34.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:34.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:22:34.796 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [12,44) [v2:127.0.0.1:6810/530496408,v1:127.0.0.1:6811/530496408] [v2:127.0.0.1:6812/530496408,v1:127.0.0.1:6813/530496408] exists,up d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:22:34.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:34.796 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:34.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:34.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:22:34.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:22:34.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:22:34.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:22:34.797 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:22:34.798 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:22:34.798 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:22:34.798 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:22:34.798 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:22:34.798 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:22:34.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:22:34.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:22:34.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:22:34.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:22:34.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:22:34.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:22:35.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:22:35.113 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:35.113 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:22:35.113 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:22:35.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:22:35.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:35.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:22:35.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-23T08:22:35.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-23T08:22:35.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788' 2026-02-23T08:22:35.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:35.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:22:35.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-23T08:22:35.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-23T08:22:35.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-210453397507' 2026-02-23T08:22:35.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:35.275 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:22:35.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444042 2026-02-23T08:22:35.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444042 2026-02-23T08:22:35.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-73014444042' 2026-02-23T08:22:35.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:35.356 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:22:35.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757254 2026-02-23T08:22:35.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757254 2026-02-23T08:22:35.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-73014444042 3-163208757254' 2026-02-23T08:22:35.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:35.433 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803788 2026-02-23T08:22:35.433 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:35.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:22:35.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-23T08:22:35.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:35.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803788 2026-02-23T08:22:35.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-23T08:22:35.437 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803788 2026-02-23T08:22:35.437 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:35.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-23T08:22:35.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:35.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-210453397507 2026-02-23T08:22:35.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:35.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:22:35.657 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397507 2026-02-23T08:22:35.658 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:35.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397507 2026-02-23T08:22:35.660 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 210453397507' 2026-02-23T08:22:35.660 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 210453397507 2026-02-23T08:22:35.660 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:22:35.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-23T08:22:35.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:35.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444042 2026-02-23T08:22:35.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:35.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:22:35.882 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444042 2026-02-23T08:22:35.882 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:35.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444042 2026-02-23T08:22:35.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444042' 2026-02-23T08:22:35.883 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444042 2026-02-23T08:22:35.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:22:36.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444042 -lt 73014444042 2026-02-23T08:22:36.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:36.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-163208757254 2026-02-23T08:22:36.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:36.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:22:36.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-163208757254 2026-02-23T08:22:36.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:36.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757254 2026-02-23T08:22:36.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 163208757254' 2026-02-23T08:22:36.114 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 163208757254 2026-02-23T08:22:36.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:36.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757254 -lt 163208757254 2026-02-23T08:22:36.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:22:36.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:36.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:22:36.641 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:22:36.857 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:22:36.857 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:22:36.858 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:36.858 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:37.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:22:37.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:22:37.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:22:37.145 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:22:37.146 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:22:37.500 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:22:37.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-23T08:22:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-0-10 fail 2026-02-23T08:22:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:22:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:22:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-0-10 2026-02-23T08:22:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-23T08:22:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-23T08:22:37.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-74343-0-10 td/test-erasure-eio/COPY 2026-02-23T08:22:37.533 INFO:tasks.workunit.client.0.trial186.stderr:error getting pool-jerasure/obj-size-74343-0-10: (5) Input/output error 2026-02-23T08:22:37.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-23T08:22:37.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:22:37.537 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74343-0-0 2026-02-23T08:22:37.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74343-0-0 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74343-0-0 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:22:37.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74343-0-0 td/test-erasure-eio/ORIGINAL 2026-02-23T08:22:37.557 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74343-0-0 td/test-erasure-eio 0 0 set 2026-02-23T08:22:37.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-0-0 2026-02-23T08:22:37.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:22:37.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:22:37.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:22:37.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-23T08:22:37.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-0-0 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-0-0 2026-02-23T08:22:37.559 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-0-0 2026-02-23T08:22:37.560 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:22:37.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:22:37.774 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:37.774 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:22:37.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:22:37.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:22:37.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:22:37.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-23T08:22:37.775 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:22:38.063 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:22:38.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:22:38.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-23T08:22:38.073 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-23T08:22:38.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:38.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:22:38.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:22:38.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-23T08:22:38.075 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:38.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:38.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:38.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:38.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:22:38.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:22:38.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-23T08:22:38.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:22:38.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:38.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:38.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-23T08:22:38.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:38.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:22:38.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:38.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:38.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:22:38.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:22:38.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:22:38.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-23T08:22:38.649 INFO:tasks.workunit.client.0.trial186.stderr:start osd.3 2026-02-23T08:22:38.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:38.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-23T08:22:38.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-23T08:22:38.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:22:38.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:22:38.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:22:38.667 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:38.656+0000 7f561455da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:38.669 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:38.660+0000 7f561455da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:38.669 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:38.660+0000 7f561455da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-23T08:22:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:22:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:38.872 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:38.873 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:22:38.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:38.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:39.042 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:39.032+0000 7f561455da40 -1 Falling back to public interface 2026-02-23T08:22:39.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:39.626 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:39.616+0000 7f561455da40 -1 osd.3 54 log_to_monitors true 2026-02-23T08:22:40.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:40.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:40.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:40.087 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:40.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:40.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:40.292 INFO:tasks.workunit.client.0.trial186.stderr:osd.3 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [38,54) [v2:127.0.0.1:6826/3918274502,v1:127.0.0.1:6827/3918274502] [v2:127.0.0.1:6828/3918274502,v1:127.0.0.1:6829/3918274502] exists,up 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:22:40.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:40.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:40.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:40.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:22:40.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:22:40.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:22:40.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:22:40.293 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:22:40.294 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:22:40.294 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:22:40.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:22:40.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:22:40.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:22:40.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:22:40.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:22:40.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:22:40.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:22:40.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:22:40.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:22:40.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:22:40.626 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:40.626 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:22:40.626 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:22:40.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:22:40.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:40.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:22:40.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-23T08:22:40.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-23T08:22:40.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-23T08:22:40.708 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:40.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:22:40.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-23T08:22:40.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-23T08:22:40.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-210453397510' 2026-02-23T08:22:40.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:40.790 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:22:40.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444045 2026-02-23T08:22:40.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444045 2026-02-23T08:22:40.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-73014444045' 2026-02-23T08:22:40.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:40.867 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:22:40.950 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-23T08:22:40.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-23T08:22:40.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-73014444045 3-253403070467' 2026-02-23T08:22:40.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:40.951 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-23T08:22:40.951 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:40.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:22:40.953 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-23T08:22:40.953 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:40.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-23T08:22:40.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-23T08:22:40.954 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803791 2026-02-23T08:22:40.955 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:41.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803791 2026-02-23T08:22:41.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:22:42.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:22:42.178 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:42.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-23T08:22:42.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:42.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-210453397510 2026-02-23T08:22:42.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:42.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:22:42.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397510 2026-02-23T08:22:42.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:42.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397510 2026-02-23T08:22:42.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-23T08:22:42.404 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 210453397510 2026-02-23T08:22:42.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:22:42.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-23T08:22:42.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:42.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444045 2026-02-23T08:22:42.634 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:42.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:22:42.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444045 2026-02-23T08:22:42.636 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:42.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444045 2026-02-23T08:22:42.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444045' 2026-02-23T08:22:42.637 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444045 2026-02-23T08:22:42.637 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:22:42.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444046 -lt 73014444045 2026-02-23T08:22:42.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:42.856 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-253403070467 2026-02-23T08:22:42.857 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:42.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:22:42.859 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070467 2026-02-23T08:22:42.859 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:42.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=253403070467 2026-02-23T08:22:42.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 253403070467' 2026-02-23T08:22:42.860 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 253403070467 2026-02-23T08:22:42.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:43.069 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-23T08:22:43.070 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:22:43.070 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:43.070 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:22:43.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:22:43.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:22:43.579 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:22:43.579 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:43.580 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:43.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:22:43.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:22:43.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:22:43.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:22:43.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:22:44.190 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:22:44.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-0-0 2026-02-23T08:22:44.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:22:44.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:22:44.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-0-0 2026-02-23T08:22:44.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:22:44.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:22:44.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-0-0 td/test-erasure-eio/COPY 2026-02-23T08:22:44.223 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:22:44.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:22:44.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-23T08:22:44.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-23T08:22:44.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74343-0-0 td/test-erasure-eio 1 0 set 2026-02-23T08:22:44.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-0-0 2026-02-23T08:22:44.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:22:44.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:22:44.228 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-0-0 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:22:44.229 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-0-0 2026-02-23T08:22:44.230 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-0-0 2026-02-23T08:22:44.230 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:22:44.447 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:22:44.447 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:44.447 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:22:44.448 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:22:44.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:22:44.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:22:44.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-23T08:22:44.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:22:44.758 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:22:44.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:22:44.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-23T08:22:44.770 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:22:44.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:22:44.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:22:44.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:22:44.773 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:44.773 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:44.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:44.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:44.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:22:45.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:45.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:45.380 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:45.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:22:45.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:22:45.383 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:22:45.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:45.384 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:22:45.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:22:45.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:22:45.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:22:45.384 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:22:45.397 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:45.388+0000 7fb52b666a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:45.398 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:45.388+0000 7fb52b666a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:45.399 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:45.388+0000 7fb52b666a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:45.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:22:45.828 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:45.830 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:45.820+0000 7fb52b666a40 -1 Falling back to public interface 2026-02-23T08:22:46.407 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:46.396+0000 7fb52b666a40 -1 osd.1 62 log_to_monitors true 2026-02-23T08:22:46.829 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:46.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:46.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:46.830 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:46.830 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:46.831 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:22:47.039 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [49,62) [v2:127.0.0.1:6810/2358055470,v1:127.0.0.1:6811/2358055470] [v2:127.0.0.1:6812/2358055470,v1:127.0.0.1:6813/2358055470] exists,up d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:22:47.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:47.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:47.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:47.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:22:47.040 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:22:47.041 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:22:47.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:22:47.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:22:47.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:22:47.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:22:47.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:22:47.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:22:47.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:22:47.373 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:47.374 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:22:47.374 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:22:47.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:22:47.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:47.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:22:47.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-23T08:22:47.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-23T08:22:47.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794' 2026-02-23T08:22:47.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:47.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:22:47.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-23T08:22:47.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-23T08:22:47.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-292057776131' 2026-02-23T08:22:47.531 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:47.531 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:22:47.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444049 2026-02-23T08:22:47.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444049 2026-02-23T08:22:47.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-292057776131 2-73014444049' 2026-02-23T08:22:47.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:47.610 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:22:47.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070470 2026-02-23T08:22:47.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070470 2026-02-23T08:22:47.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-292057776131 2-73014444049 3-253403070470' 2026-02-23T08:22:47.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:47.692 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803794 2026-02-23T08:22:47.693 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:47.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:22:47.694 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-23T08:22:47.694 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:47.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803794 2026-02-23T08:22:47.696 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-23T08:22:47.696 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803794 2026-02-23T08:22:47.696 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:47.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803794 2026-02-23T08:22:47.929 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:22:48.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:22:48.931 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:49.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803794 2026-02-23T08:22:49.150 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:22:50.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-23T08:22:50.151 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:50.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-23T08:22:50.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:50.363 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-292057776131 2026-02-23T08:22:50.363 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:50.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:22:50.365 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776131 2026-02-23T08:22:50.365 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:50.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776131 2026-02-23T08:22:50.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-23T08:22:50.367 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 292057776131 2026-02-23T08:22:50.367 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:22:50.585 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-23T08:22:50.585 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:50.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444049 2026-02-23T08:22:50.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:50.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:22:50.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444049 2026-02-23T08:22:50.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:50.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444049 2026-02-23T08:22:50.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444049' 2026-02-23T08:22:50.589 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444049 2026-02-23T08:22:50.590 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:22:50.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444049 -lt 73014444049 2026-02-23T08:22:50.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:50.801 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-253403070470 2026-02-23T08:22:50.801 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:50.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:22:50.803 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070470 2026-02-23T08:22:50.803 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:50.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=253403070470 2026-02-23T08:22:50.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 253403070470' 2026-02-23T08:22:50.805 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 253403070470 2026-02-23T08:22:50.805 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:51.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070470 -lt 253403070470 2026-02-23T08:22:51.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:22:51.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:51.027 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:51.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:22:51.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:22:51.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:22:51.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:22:51.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:22:51.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:22:51.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:22:51.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:22:51.525 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:22:51.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:22:51.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:51.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:51.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:22:51.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:22:51.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:22:51.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:22:51.820 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:22:52.138 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:22:52.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-23T08:22:52.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-0-0 fail 2026-02-23T08:22:52.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:22:52.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:22:52.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-0-0 2026-02-23T08:22:52.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-23T08:22:52.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-23T08:22:52.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-74343-0-0 td/test-erasure-eio/COPY 2026-02-23T08:22:52.176 INFO:tasks.workunit.client.0.trial186.stderr:error getting pool-jerasure/obj-size-74343-0-0: (5) Input/output error 2026-02-23T08:22:52.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-23T08:22:52.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:22:52.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-23T08:22:52.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-23T08:22:52.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74343-0-256 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74343-0-256 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:22:52.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74343-0-256 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:22:52.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74343-0-256 td/test-erasure-eio/ORIGINAL 2026-02-23T08:22:52.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74343-0-256 td/test-erasure-eio 0 256 add 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-0-256 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-23T08:22:52.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:22:52.206 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-0-256 2026-02-23T08:22:52.206 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:22:52.206 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-0-256 2026-02-23T08:22:52.206 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-0-256 2026-02-23T08:22:52.206 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:22:52.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:22:52.419 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:52.419 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:22:52.420 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:22:52.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:22:52.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:22:52.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-23T08:22:52.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:22:52.711 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:22:52.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-23T08:22:52.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:52.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:22:52.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:22:52.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-23T08:22:52.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:22:52.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:52.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:53.034 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:53.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:53.204 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:53.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:53.206 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:22:53.206 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:22:53.206 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:22:53.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-23T08:22:53.207 INFO:tasks.workunit.client.0.trial186.stderr:start osd.3 2026-02-23T08:22:53.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:53.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-23T08:22:53.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-23T08:22:53.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:22:53.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:22:53.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:22:53.223 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:53.212+0000 7fa2974a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:53.224 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:53.212+0000 7fa2974a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:53.225 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:53.216+0000 7fa2974a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:53.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-23T08:22:53.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:53.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:22:53.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:53.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:53.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:53.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:53.429 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:22:53.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:53.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:53.574 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:53.564+0000 7fa2974a4a40 -1 Falling back to public interface 2026-02-23T08:22:53.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:22:54.246 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:54.236+0000 7fa2974a4a40 -1 osd.3 72 log_to_monitors true 2026-02-23T08:22:54.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:22:54.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:54.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:22:54.649 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:54.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:54.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:54.862 INFO:tasks.workunit.client.0.trial186.stderr:osd.3 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,72) [v2:127.0.0.1:6826/4269697654,v1:127.0.0.1:6827/4269697654] [v2:127.0.0.1:6828/4269697654,v1:127.0.0.1:6829/4269697654] exists,up 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:22:54.863 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:22:54.864 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:22:54.864 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:22:54.864 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:22:54.864 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:22:54.865 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:22:54.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:22:54.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:22:54.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:22:54.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:22:54.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:22:54.968 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:22:55.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:22:55.186 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:22:55.187 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:22:55.187 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:22:55.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:22:55.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:55.187 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:22:55.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803798 2026-02-23T08:22:55.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803798 2026-02-23T08:22:55.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798' 2026-02-23T08:22:55.265 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:55.265 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:22:55.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776134 2026-02-23T08:22:55.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776134 2026-02-23T08:22:55.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-292057776134' 2026-02-23T08:22:55.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:55.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:22:55.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444052 2026-02-23T08:22:55.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444052 2026-02-23T08:22:55.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-292057776134 2-73014444052' 2026-02-23T08:22:55.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:22:55.429 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:22:55.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481795 2026-02-23T08:22:55.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481795 2026-02-23T08:22:55.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-292057776134 2-73014444052 3-330712481795' 2026-02-23T08:22:55.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:55.507 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803798 2026-02-23T08:22:55.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:55.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:22:55.509 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803798 2026-02-23T08:22:55.509 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:55.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803798 2026-02-23T08:22:55.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803798' 2026-02-23T08:22:55.511 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803798 2026-02-23T08:22:55.511 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:22:55.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803798 -lt 25769803798 2026-02-23T08:22:55.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:55.737 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-292057776134 2026-02-23T08:22:55.737 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:55.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:22:55.739 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776134 2026-02-23T08:22:55.739 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:55.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776134 2026-02-23T08:22:55.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 292057776134' 2026-02-23T08:22:55.741 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 292057776134 2026-02-23T08:22:55.741 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:22:55.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-23T08:22:55.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:55.960 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444052 2026-02-23T08:22:55.960 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:55.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:22:55.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444052 2026-02-23T08:22:55.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:55.963 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444052 2026-02-23T08:22:55.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444052' 2026-02-23T08:22:55.964 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444052 2026-02-23T08:22:55.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:22:56.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444052 -lt 73014444052 2026-02-23T08:22:56.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:22:56.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-330712481795 2026-02-23T08:22:56.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:22:56.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:22:56.183 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-330712481795 2026-02-23T08:22:56.183 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:22:56.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=330712481795 2026-02-23T08:22:56.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 330712481795' 2026-02-23T08:22:56.185 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 330712481795 2026-02-23T08:22:56.185 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:56.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 330712481795 2026-02-23T08:22:56.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:22:57.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:22:57.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:22:57.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-23T08:22:57.622 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:22:57.623 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:57.623 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:57.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:22:57.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:22:57.919 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:22:57.919 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:22:57.919 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:22:57.919 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:22:57.919 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:22:57.919 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:22:58.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:22:58.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:22:58.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:22:58.136 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:22:58.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:22:58.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:22:58.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:22:58.430 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-23T08:22:58.432 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:22:58.432 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:22:58.432 INFO:tasks.workunit.client.0.trial186.stderr:256 bytes copied, 2.0879e-05 s, 12.3 MB/s 2026-02-23T08:22:58.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-23T08:22:58.433 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:22:58.434 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:22:58.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:22:58.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:22:58.434 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:22:58.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:22:58.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:58.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:22:58.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:22:58.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-23T08:22:58.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:22:58.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:58.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:22:59.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:22:59.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:22:59.037 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:22:59.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:22:59.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-23T08:22:59.040 INFO:tasks.workunit.client.0.trial186.stderr:start osd.3 2026-02-23T08:22:59.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:22:59.040 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-23T08:22:59.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-23T08:22:59.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:22:59.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:22:59.040 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:22:59.055 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:59.044+0000 7fbde99e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:59.057 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:59.048+0000 7fbde99e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:59.058 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:59.048+0000 7fbde99e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:22:59.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:22:59.442 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:22:59.432+0000 7fbde99e6a40 -1 Falling back to public interface 2026-02-23T08:22:59.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:00.051 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:00.040+0000 7fbde99e6a40 -1 osd.3 79 log_to_monitors true 2026-02-23T08:23:00.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:00.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:00.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:00.473 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:00.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:00.473 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:23:00.687 INFO:tasks.workunit.client.0.trial186.stderr:osd.3 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [77,79) [v2:127.0.0.1:6826/1900768523,v1:127.0.0.1:6827/1900768523] [v2:127.0.0.1:6828/1900768523,v1:127.0.0.1:6829/1900768523] exists,up 0edbfb58-b060-4ce0-b7a2-481902e6f5c3 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:23:00.688 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:23:00.689 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:23:00.689 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:23:00.689 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:23:00.689 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:23:00.689 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:23:00.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:23:00.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:23:00.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:23:00.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:23:00.789 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:23:00.789 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:23:01.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:23:01.001 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:01.001 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:23:01.001 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:23:01.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:23:01.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:01.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:23:01.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803801 2026-02-23T08:23:01.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803801 2026-02-23T08:23:01.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801' 2026-02-23T08:23:01.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:01.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:23:01.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776137 2026-02-23T08:23:01.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776137 2026-02-23T08:23:01.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-292057776137' 2026-02-23T08:23:01.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:01.162 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:23:01.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444055 2026-02-23T08:23:01.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444055 2026-02-23T08:23:01.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-292057776137 2-73014444055' 2026-02-23T08:23:01.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:01.244 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:23:01.325 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-23T08:23:01.325 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-23T08:23:01.325 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-292057776137 2-73014444055 3-356482285571' 2026-02-23T08:23:01.325 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:01.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803801 2026-02-23T08:23:01.325 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:01.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:23:01.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803801 2026-02-23T08:23:01.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:01.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803801 2026-02-23T08:23:01.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803801' 2026-02-23T08:23:01.328 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803801 2026-02-23T08:23:01.328 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:01.546 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803801 -lt 25769803801 2026-02-23T08:23:01.546 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:01.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-292057776137 2026-02-23T08:23:01.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:01.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:23:01.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776137 2026-02-23T08:23:01.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:01.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776137 2026-02-23T08:23:01.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 292057776137' 2026-02-23T08:23:01.550 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 292057776137 2026-02-23T08:23:01.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:23:01.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776137 -lt 292057776137 2026-02-23T08:23:01.769 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:01.770 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444055 2026-02-23T08:23:01.770 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:01.771 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:23:01.772 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444055 2026-02-23T08:23:01.772 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:01.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444055 2026-02-23T08:23:01.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444055' 2026-02-23T08:23:01.773 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444055 2026-02-23T08:23:01.774 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:23:01.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444056 -lt 73014444055 2026-02-23T08:23:01.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:01.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-356482285571 2026-02-23T08:23:01.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:01.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:23:01.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285571 2026-02-23T08:23:01.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:01.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285571 2026-02-23T08:23:02.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 356482285571' 2026-02-23T08:23:02.000 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 356482285571 2026-02-23T08:23:02.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:23:02.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-23T08:23:02.225 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:23:02.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:02.226 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:02.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:23:02.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:23:02.509 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:23:02.509 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:23:02.509 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:23:02.509 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:23:02.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:23:02.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:23:02.714 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:23:02.715 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:23:02.715 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:02.715 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:03.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:23:03.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:23:03.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:23:03.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:23:03.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:23:03.318 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:23:03.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-0-256 2026-02-23T08:23:03.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:23:03.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:23:03.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-0-256 2026-02-23T08:23:03.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:23:03.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:23:03.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-0-256 td/test-erasure-eio/COPY 2026-02-23T08:23:03.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:23:03.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:23:03.354 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-23T08:23:03.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-23T08:23:03.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74343-0-256 td/test-erasure-eio 1 256 add 2026-02-23T08:23:03.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-0-256 2026-02-23T08:23:03.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:23:03.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:23:03.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:23:03.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-23T08:23:03.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:23:03.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-23T08:23:03.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:23:03.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-23T08:23:03.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:23:03.358 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-0-256 2026-02-23T08:23:03.358 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:23:03.358 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-0-256 2026-02-23T08:23:03.358 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-0-256 2026-02-23T08:23:03.358 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:23:03.567 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:23:03.567 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:03.567 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:23:03.568 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:23:03.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:23:03.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:23:03.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-23T08:23:03.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:23:03.884 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:23:03.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-23T08:23:03.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:23:03.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:23:03.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:03.897 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:03.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:03.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:03.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:04.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:04.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:04.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:23:04.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:23:04.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:23:04.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:23:04.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:04.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:04.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:23:04.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:04.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:04.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:04.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:04.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:04.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:04.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:04.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:23:04.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:23:04.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:23:04.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:23:04.375 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:23:04.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:04.375 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:23:04.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:23:04.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:23:04.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:23:04.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:23:04.389 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:04.380+0000 7f7721dcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:04.391 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:04.380+0000 7f7721dcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:04.392 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:04.380+0000 7f7721dcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:04.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:23:04.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:04.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:23:04.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:04.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:04.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:04.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:04.594 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:23:04.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:04.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:04.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:04.818 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:04.808+0000 7f7721dcfa40 -1 Falling back to public interface 2026-02-23T08:23:05.445 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:05.436+0000 7f7721dcfa40 -1 osd.1 86 log_to_monitors true 2026-02-23T08:23:05.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:05.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:05.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:05.807 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:05.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:05.807 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:06.019 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 92 up_thru 92 down_at 89 last_clean_interval [68,86) [v2:127.0.0.1:6810/663588741,v1:127.0.0.1:6811/663588741] [v2:127.0.0.1:6812/663588741,v1:127.0.0.1:6813/663588741] exists,up d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:23:06.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:06.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:06.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:06.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:23:06.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:23:06.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:23:06.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:23:06.021 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:23:06.021 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:23:06.021 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:23:06.021 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:23:06.021 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:23:06.021 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:23:06.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:23:06.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:23:06.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:23:06.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:23:06.127 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:23:06.127 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:23:06.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:23:06.347 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:06.347 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:23:06.347 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:23:06.347 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:23:06.347 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:06.347 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:23:06.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-23T08:23:06.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-23T08:23:06.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804' 2026-02-23T08:23:06.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:06.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:23:06.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991235 2026-02-23T08:23:06.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991235 2026-02-23T08:23:06.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-395136991235' 2026-02-23T08:23:06.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:06.506 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:23:06.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444059 2026-02-23T08:23:06.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444059 2026-02-23T08:23:06.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-395136991235 2-73014444059' 2026-02-23T08:23:06.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:06.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:23:06.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285574 2026-02-23T08:23:06.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285574 2026-02-23T08:23:06.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-395136991235 2-73014444059 3-356482285574' 2026-02-23T08:23:06.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:06.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803804 2026-02-23T08:23:06.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:06.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:23:06.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-23T08:23:06.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:06.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803804 2026-02-23T08:23:06.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-23T08:23:06.671 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803804 2026-02-23T08:23:06.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:06.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803802 -lt 25769803804 2026-02-23T08:23:06.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:23:07.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:23:07.890 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:08.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-23T08:23:08.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:08.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-395136991235 2026-02-23T08:23:08.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:08.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:23:08.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-395136991235 2026-02-23T08:23:08.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:08.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991235 2026-02-23T08:23:08.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 395136991235' 2026-02-23T08:23:08.104 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 395136991235 2026-02-23T08:23:08.104 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:23:08.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-23T08:23:08.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:08.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444059 2026-02-23T08:23:08.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:08.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:23:08.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444059 2026-02-23T08:23:08.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:08.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444059 2026-02-23T08:23:08.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444059' 2026-02-23T08:23:08.337 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444059 2026-02-23T08:23:08.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:23:08.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444059 -lt 73014444059 2026-02-23T08:23:08.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:08.560 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-356482285574 2026-02-23T08:23:08.560 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:08.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:23:08.562 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285574 2026-02-23T08:23:08.562 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:08.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285574 2026-02-23T08:23:08.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 356482285574' 2026-02-23T08:23:08.564 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 356482285574 2026-02-23T08:23:08.564 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:23:08.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285574 -lt 356482285574 2026-02-23T08:23:08.793 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:23:08.793 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:08.793 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:09.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:23:09.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:23:09.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:23:09.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:23:09.089 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:23:09.090 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:23:09.090 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:23:09.090 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:23:09.304 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:23:09.305 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:23:09.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:09.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:09.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:23:09.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:23:09.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:23:09.593 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-23T08:23:09.596 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:23:09.596 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:23:09.596 INFO:tasks.workunit.client.0.trial186.stderr:256 bytes copied, 3.8682e-05 s, 6.6 MB/s 2026-02-23T08:23:09.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:09.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:09.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:09.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:09.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:09.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:09.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:09.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:23:09.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:23:09.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:23:09.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:23:09.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:09.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:10.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:23:10.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:10.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:23:10.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:23:10.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:23:10.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:10.192 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:10.193 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:10.194 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:23:10.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:23:10.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:23:10.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:23:10.211 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:10.200+0000 7f8958cb1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:10.213 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:10.204+0000 7f8958cb1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:10.213 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:10.204+0000 7f8958cb1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:10.416 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:23:10.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:10.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:10.598 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:10.588+0000 7f8958cb1a40 -1 Falling back to public interface 2026-02-23T08:23:10.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:11.248 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:11.236+0000 7f8958cb1a40 -1 osd.1 93 log_to_monitors true 2026-02-23T08:23:11.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:11.637 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:11.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:11.638 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:11.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:11.638 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:11.852 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 97 up_thru 97 down_at 94 last_clean_interval [92,93) [v2:127.0.0.1:6810/2977478579,v1:127.0.0.1:6811/2977478579] [v2:127.0.0.1:6812/2977478579,v1:127.0.0.1:6813/2977478579] exists,up d55d371e-5310-4a90-af6f-cbd9544a3f63 2026-02-23T08:23:11.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:11.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:11.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:11.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:23:11.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:23:11.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:23:11.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:23:11.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:23:11.854 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:23:11.854 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:23:11.854 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:23:11.854 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:23:11.854 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:23:11.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:23:11.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:23:11.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:23:11.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:23:11.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:23:11.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:23:12.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:23:12.177 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:12.177 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:23:12.177 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:23:12.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:23:12.177 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:12.178 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:23:12.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-23T08:23:12.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-23T08:23:12.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807' 2026-02-23T08:23:12.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:12.258 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:23:12.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=416611827715 2026-02-23T08:23:12.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 416611827715 2026-02-23T08:23:12.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-416611827715' 2026-02-23T08:23:12.338 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:12.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:23:12.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444062 2026-02-23T08:23:12.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444062 2026-02-23T08:23:12.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-416611827715 2-73014444062' 2026-02-23T08:23:12.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:12.416 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:23:12.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285577 2026-02-23T08:23:12.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285577 2026-02-23T08:23:12.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-416611827715 2-73014444062 3-356482285577' 2026-02-23T08:23:12.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:12.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803807 2026-02-23T08:23:12.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:12.496 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:23:12.497 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-23T08:23:12.497 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:12.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803807 2026-02-23T08:23:12.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-23T08:23:12.499 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803807 2026-02-23T08:23:12.499 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:12.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-23T08:23:12.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:23:13.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:23:13.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:13.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-23T08:23:13.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:13.947 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-416611827715 2026-02-23T08:23:13.948 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:13.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:23:13.949 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-416611827715 2026-02-23T08:23:13.949 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:13.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=416611827715 2026-02-23T08:23:13.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 416611827715' 2026-02-23T08:23:13.951 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 416611827715 2026-02-23T08:23:13.951 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:23:14.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-23T08:23:14.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:14.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444062 2026-02-23T08:23:14.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:14.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:23:14.183 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444062 2026-02-23T08:23:14.183 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:14.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444062 2026-02-23T08:23:14.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444062' 2026-02-23T08:23:14.185 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 73014444062 2026-02-23T08:23:14.185 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:23:14.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444062 -lt 73014444062 2026-02-23T08:23:14.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:14.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-356482285577 2026-02-23T08:23:14.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:14.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:23:14.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285577 2026-02-23T08:23:14.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285577 2026-02-23T08:23:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 356482285577' 2026-02-23T08:23:14.405 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 356482285577 2026-02-23T08:23:14.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:23:14.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285577 -lt 356482285577 2026-02-23T08:23:14.624 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:23:14.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:14.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:14.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:23:14.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:23:14.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:23:14.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:23:14.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:23:14.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:23:14.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:23:14.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:23:15.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:23:15.118 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:23:15.118 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:15.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:15.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:23:15.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:23:15.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:23:15.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:23:15.411 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:23:15.734 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-0-256 fail 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-0-256 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-23T08:23:15.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-74343-0-256 td/test-erasure-eio/COPY 2026-02-23T08:23:15.770 INFO:tasks.workunit.client.0.trial186.stderr:error getting pool-jerasure/obj-size-74343-0-256: (5) Input/output error 2026-02-23T08:23:15.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-23T08:23:15.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:23:15.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:23:15.773 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:23:15.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:23:15.989 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:23:16.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:23:16.297 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:23:16.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:23:16.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:23:16.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:23:16.308 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:23:16.308 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:16.308 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:16.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:16.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:16.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:16.620 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:16.621 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:23:16.622 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:23:16.622 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:23:16.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:23:16.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:23:16.625 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:23:16.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:23:16.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:23:16.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:23:16.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:23:16.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:23:16.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:23:16.629 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:23:16.653 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:23:16.653 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:16.653 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:16.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:23:16.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:16.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:16.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:16.658 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:23:16.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:23:16.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:23:16.660 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:23:16.660 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:23:16.661 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:23:16.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:23:16.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:23:16.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:23:16.662 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:23:16.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:23:16.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:23:16.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:23:16.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:23:16.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:16.666 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:16.666 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:23:16.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:23:16.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:23:16.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:23:16.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:23:16.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:16.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:16.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:23:16.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:23:16.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:23:16.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:23:16.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:23:16.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:23:16.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:23:16.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:23:16.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:23:16.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:23:16.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:23:16.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:23:16.693 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:23:16.693 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:16.693 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:16.694 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:16.694 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:16.694 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:16.694 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:16.694 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:23:16.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:23:16.726 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:23:16.726 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:23:16.726 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:23:16.726 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:16.728 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:23:16.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:23:16.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:23:16.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:23:16.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:23:16.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:23:16.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:23:16.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:23:16.786 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:23:16.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:23:16.787 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:23:16.788 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:23:16.788 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:16.788 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:16.788 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:23:16.788 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:23:16.788 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:23:16.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:23:16.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:23:16.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:23:16.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:23:16.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:23:16.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:23:16.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:23:16.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:23:16.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:16.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:16.959 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:16.959 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:16.959 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:16.959 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:16.960 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:23:16.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:23:16.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:23:16.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:23:17.091 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:23:17.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:23:18.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:23:18.153 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:23:18.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:23:18.161 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:23:16.704+0000 7f3d626a7ec0 0 load: jerasure load: lrc 2026-02-23T08:23:18.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-23T08:23:18.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-23T08:23:18.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-23T08:23:18.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-23T08:23:18.162 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:23:18.163 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-23T08:23:18.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:18.167 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:18.168 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:18.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:18.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:23:18.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:23:18.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:23:18.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:23:18.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dbe79d34-486e-4752-9ee8-5855f9980595 2026-02-23T08:23:18.171 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 dbe79d34-486e-4752-9ee8-5855f9980595 2026-02-23T08:23:18.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dbe79d34-486e-4752-9ee8-5855f9980595' 2026-02-23T08:23:18.171 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:23:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB2Dpxpk/pZChAAdYI0udpS097G0pfg7satWQ== 2026-02-23T08:23:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB2Dpxpk/pZChAAdYI0udpS097G0pfg7satWQ=="}' 2026-02-23T08:23:18.179 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dbe79d34-486e-4752-9ee8-5855f9980595 -i td/test-erasure-eio/0/new.json 2026-02-23T08:23:18.296 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:23:18.304 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:23:18.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB2Dpxpk/pZChAAdYI0udpS097G0pfg7satWQ== --osd-uuid dbe79d34-486e-4752-9ee8-5855f9980595 2026-02-23T08:23:18.321 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:18.312+0000 7fe0a63f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:18.323 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:18.312+0000 7fe0a63f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:18.324 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:18.312+0000 7fe0a63f0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:18.324 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:18.312+0000 7fe0a63f0a40 -1 bdev(0x561c3f010400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:23:18.324 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:18.312+0000 7fe0a63f0a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:23:19.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:23:19.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:23:19.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:23:19.016 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:23:19.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:23:19.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:23:19.134 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:23:19.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:19.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:23:19.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:23:19.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:23:19.149 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:19.140+0000 7fc3da7cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:19.151 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:19.140+0000 7fc3da7cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:19.151 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:19.140+0000 7fc3da7cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:19.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:23:19.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:19.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:23:19.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:19.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:19.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:19.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:19.287 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:23:19.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:19.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:23:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:19.610 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:19.600+0000 7fc3da7cca40 -1 Falling back to public interface 2026-02-23T08:23:20.218 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:20.208+0000 7fc3da7cca40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:23:20.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:20.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:20.398 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:23:20.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:20.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:20.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:23:20.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:21.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:21.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:21.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:23:21.610 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:23:21.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:21.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:23:21.822 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1364604687,v1:127.0.0.1:6803/1364604687] [v2:127.0.0.1:6804/1364604687,v1:127.0.0.1:6805/1364604687] exists,up dbe79d34-486e-4752-9ee8-5855f9980595 2026-02-23T08:23:21.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:21.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:21.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:21.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:21.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:23:21.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:23:21.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:23:21.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:23:21.827 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a0e80d37-0449-4cd6-ad23-f2406a6c2017' 2026-02-23T08:23:21.827 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:23:21.827 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:23:21.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB5DpxpgvG4MRAANAJSQccH+wc5rRfdBD4ekA== 2026-02-23T08:23:21.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB5DpxpgvG4MRAANAJSQccH+wc5rRfdBD4ekA=="}' 2026-02-23T08:23:21.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a0e80d37-0449-4cd6-ad23-f2406a6c2017 -i td/test-erasure-eio/1/new.json 2026-02-23T08:23:22.067 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:23:22.078 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:23:22.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB5DpxpgvG4MRAANAJSQccH+wc5rRfdBD4ekA== --osd-uuid a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:23:22.102 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:22.092+0000 7f4b87f0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:22.104 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:22.092+0000 7f4b87f0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:22.105 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:22.096+0000 7f4b87f0aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:22.105 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:22.096+0000 7f4b87f0aa40 -1 bdev(0x55d0bc852400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:23:22.106 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:22.096+0000 7f4b87f0aa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:23:22.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:23:22.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:23:22.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:23:22.721 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:23:22.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:23:23.007 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:23:23.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:23:23.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:23.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:23:23.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:23:23.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:23:23.031 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:23.020+0000 7fa3d531ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:23.033 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:23.024+0000 7fa3d531ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:23.034 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:23.024+0000 7fa3d531ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:23.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:23:23.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:23.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:23:23.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:23.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:23.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:23.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:23.226 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:23:23.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:23.226 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:23.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:23.474 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:23.464+0000 7fa3d531ea40 -1 Falling back to public interface 2026-02-23T08:23:24.050 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:24.040+0000 7fa3d531ea40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:23:24.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:24.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:24.438 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:23:24.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:24.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:24.439 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:24.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:25.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:25.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:25.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:23:25.658 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:23:25.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:25.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:25.870 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/499730191,v1:127.0.0.1:6811/499730191] [v2:127.0.0.1:6812/499730191,v1:127.0.0.1:6813/499730191] exists,up a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:23:25.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:25.875 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:25.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:25.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:25.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:25.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:25.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:23:25.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:23:25.877 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:23:25.878 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:23:25.878 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:23:25.878 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:23:25.878 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c7aee2d9-bf35-49d7-9277-135d1aff6232' 2026-02-23T08:23:25.879 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:23:25.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB9Dpxpt5eeNBAAeJnpOKykN98PI2zE8TSTsw== 2026-02-23T08:23:25.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB9Dpxpt5eeNBAAeJnpOKykN98PI2zE8TSTsw=="}' 2026-02-23T08:23:25.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7aee2d9-bf35-49d7-9277-135d1aff6232 -i td/test-erasure-eio/2/new.json 2026-02-23T08:23:26.115 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:23:26.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:23:26.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB9Dpxpt5eeNBAAeJnpOKykN98PI2zE8TSTsw== --osd-uuid c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:23:26.153 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:26.144+0000 7fe4618b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:26.154 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:26.144+0000 7fe4618b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:26.155 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:26.144+0000 7fe4618b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:26.156 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:26.144+0000 7fe4618b1a40 -1 bdev(0x558c16986400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:23:26.156 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:26.144+0000 7fe4618b1a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:23:26.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:23:26.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:23:26.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:23:26.749 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:23:26.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:23:27.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:23:27.036 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:23:27.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:27.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:23:27.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:23:27.037 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:23:27.060 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:27.052+0000 7f3f9221ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:27.062 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:27.056+0000 7f3f9221ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:27.063 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:27.056+0000 7f3f9221ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:27.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:23:27.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:27.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:23:27.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:27.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:27.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:27.255 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:27.255 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:23:27.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:27.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:23:27.419 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:27.412+0000 7f3f9221ba40 -1 Falling back to public interface 2026-02-23T08:23:27.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:28.010 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:28.004+0000 7f3f9221ba40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:23:28.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:28.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:28.470 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:23:28.470 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:28.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:28.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:23:28.682 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1193646482,v1:127.0.0.1:6819/1193646482] [v2:127.0.0.1:6820/1193646482,v1:127.0.0.1:6821/1193646482] exists,up c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:23:28.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:28.687 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:28.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:28.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:28.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:28.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:23:28.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:23:28.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:23:28.689 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:23:28.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=026b74ee-8906-448e-88df-e2d6e2620f71 2026-02-23T08:23:28.690 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 026b74ee-8906-448e-88df-e2d6e2620f71' 2026-02-23T08:23:28.690 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 026b74ee-8906-448e-88df-e2d6e2620f71 2026-02-23T08:23:28.690 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:23:28.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCADpxpn+ddKRAAvwbsRm9oe9AfCKU7A1GlEA== 2026-02-23T08:23:28.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCADpxpn+ddKRAAvwbsRm9oe9AfCKU7A1GlEA=="}' 2026-02-23T08:23:28.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 026b74ee-8906-448e-88df-e2d6e2620f71 -i td/test-erasure-eio/3/new.json 2026-02-23T08:23:28.927 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:23:28.938 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:23:28.939 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCADpxpn+ddKRAAvwbsRm9oe9AfCKU7A1GlEA== --osd-uuid 026b74ee-8906-448e-88df-e2d6e2620f71 2026-02-23T08:23:28.963 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:28.956+0000 7fa79fbd3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:28.964 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:28.960+0000 7fa79fbd3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:28.966 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:28.960+0000 7fa79fbd3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:28.966 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:28.960+0000 7fa79fbd3a40 -1 bdev(0x55b589c04400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:23:28.966 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:28.960+0000 7fa79fbd3a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:23:29.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:23:29.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:23:29.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:23:29.609 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:23:29.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:23:29.898 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:23:29.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:23:29.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:29.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:23:29.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:23:29.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:23:29.921 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:29.912+0000 7fe07a5d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:29.922 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:29.916+0000 7fe07a5d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:29.923 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:29.916+0000 7fe07a5d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:30.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:23:30.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:30.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:23:30.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:30.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:30.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:30.125 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:23:30.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:30.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:30.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:23:30.306 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:30.300+0000 7fe07a5d1a40 -1 Falling back to public interface 2026-02-23T08:23:30.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:30.870 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:30.864+0000 7fe07a5d1a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:23:31.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:31.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:31.340 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:23:31.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:31.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:31.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:23:31.558 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/897717299,v1:127.0.0.1:6827/897717299] [v2:127.0.0.1:6828/897717299,v1:127.0.0.1:6829/897717299] exists,up 026b74ee-8906-448e-88df-e2d6e2620f71 2026-02-23T08:23:31.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:31.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:31.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:31.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:23:31.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:23:31.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:23:31.559 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:23:31.559 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:31.559 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:31.560 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:23:31.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:23:31.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:23:31.624 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:23:31.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:23:19.604+0000 7fc3da7cca40 0 load: jerasure load: lrc 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:23:31.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-23T08:23:31.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:23:31.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:23:32.345 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:23:32.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:23:33.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:23:33.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:23:33.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:23:33.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:23:33.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:23:33.358 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:23:33.358 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:23:33.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:23:33.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:23:33.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:23:33.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:23:33.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:23:33.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:23:33.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:23:33.463 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:23:33.463 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:23:33.682 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:23:33.683 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:33.683 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:23:33.683 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:23:33.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:23:33.683 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:33.683 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:23:33.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-23T08:23:33.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-23T08:23:33.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-23T08:23:33.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:33.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:23:33.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-23T08:23:33.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-23T08:23:33.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-23T08:23:33.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:33.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:23:33.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-23T08:23:33.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-23T08:23:33.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378628' 2026-02-23T08:23:33.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:33.924 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:23:33.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-23T08:23:33.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-23T08:23:33.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378628 3-115964116995' 2026-02-23T08:23:33.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:34.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-23T08:23:34.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:34.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:23:34.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-23T08:23:34.002 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:34.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-23T08:23:34.003 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803781 2026-02-23T08:23:34.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-23T08:23:34.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:34.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-23T08:23:34.225 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:23:35.227 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:23:35.227 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:35.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-23T08:23:35.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:35.449 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574852 2026-02-23T08:23:35.449 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:35.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:23:35.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-23T08:23:35.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:35.453 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574852 2026-02-23T08:23:35.454 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574852 2026-02-23T08:23:35.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-23T08:23:35.454 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:23:35.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574852 2026-02-23T08:23:35.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:35.674 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378628 2026-02-23T08:23:35.674 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:35.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:23:35.676 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378628 2026-02-23T08:23:35.676 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:35.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378628 2026-02-23T08:23:35.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-23T08:23:35.678 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 81604378628 2026-02-23T08:23:35.678 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:23:35.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378629 -lt 81604378628 2026-02-23T08:23:35.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:35.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116995 2026-02-23T08:23:35.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:35.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:23:35.896 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-23T08:23:35.896 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:35.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116995 2026-02-23T08:23:35.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-23T08:23:35.898 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 115964116995 2026-02-23T08:23:35.898 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:23:36.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-23T08:23:36.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:23:36.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:36.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:36.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:23:36.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:23:36.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:23:36.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:23:36.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:23:36.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:23:36.407 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:23:36.407 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:23:36.623 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:23:36.623 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:23:36.624 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:36.624 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74343-1-10 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74343-1-10 2026-02-23T08:23:36.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74343-1-10 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:23:36.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74343-1-10 td/test-erasure-eio/ORIGINAL 2026-02-23T08:23:36.924 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74343-1-10 td/test-erasure-eio 1 10 set 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-1-10 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-1-10 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-1-10 2026-02-23T08:23:36.925 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-1-10 2026-02-23T08:23:36.926 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:23:37.147 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:23:37.147 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:37.148 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:23:37.148 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:23:37.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:23:37.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:23:37.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-23T08:23:37.148 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:23:37.457 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:23:37.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:23:37.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-23T08:23:37.469 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:10 bytes copied, 3.237e-05 s, 309 kB/s 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:37.471 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:37.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:37.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:37.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:37.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:23:37.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:23:37.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:23:37.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:23:37.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:37.780 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:38.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:38.086 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:23:38.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:23:38.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:23:38.088 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:23:38.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:38.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:23:38.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:23:38.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:23:38.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:23:38.090 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:23:38.107 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:38.099+0000 7fe302bf5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:38.108 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:38.099+0000 7fe302bf5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:38.109 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:38.103+0000 7fe302bf5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:38.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:38.462 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:38.455+0000 7fe302bf5a40 -1 Falling back to public interface 2026-02-23T08:23:38.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:39.104 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:39.095+0000 7fe302bf5a40 -1 osd.1 36 log_to_monitors true 2026-02-23T08:23:39.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:39.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:39.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:39.517 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:39.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:39.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:39.734 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [13,36) [v2:127.0.0.1:6810/39793374,v1:127.0.0.1:6811/39793374] [v2:127.0.0.1:6812/39793374,v1:127.0.0.1:6813/39793374] exists,up a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:23:39.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:39.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:39.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:39.734 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:23:39.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:23:39.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:23:39.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:23:39.735 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:23:39.736 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:23:39.736 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:23:39.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:23:39.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:23:39.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:23:39.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:23:39.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:23:39.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:23:39.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:23:39.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:23:39.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:23:40.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:23:40.053 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:40.053 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:23:40.054 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:23:40.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:23:40.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:40.054 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:23:40.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-23T08:23:40.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-23T08:23:40.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-23T08:23:40.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:40.135 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:23:40.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659139 2026-02-23T08:23:40.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659139 2026-02-23T08:23:40.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-176093659139' 2026-02-23T08:23:40.216 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:40.216 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:23:40.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378632 2026-02-23T08:23:40.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378632 2026-02-23T08:23:40.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-176093659139 2-81604378632' 2026-02-23T08:23:40.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:40.295 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:23:40.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116998 2026-02-23T08:23:40.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116998 2026-02-23T08:23:40.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-176093659139 2-81604378632 3-115964116998' 2026-02-23T08:23:40.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:40.375 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-23T08:23:40.375 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:40.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:23:40.377 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-23T08:23:40.377 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:40.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-23T08:23:40.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-23T08:23:40.379 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803784 2026-02-23T08:23:40.379 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:40.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-23T08:23:40.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:23:41.590 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:23:41.590 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:41.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-23T08:23:41.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:41.814 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-176093659139 2026-02-23T08:23:41.814 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:41.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:23:41.816 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-176093659139 2026-02-23T08:23:41.816 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:41.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659139 2026-02-23T08:23:41.817 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 176093659139' 2026-02-23T08:23:41.818 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 176093659139 2026-02-23T08:23:41.818 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:23:42.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-23T08:23:42.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:42.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378632 2026-02-23T08:23:42.029 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:42.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:23:42.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378632 2026-02-23T08:23:42.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:42.033 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378632 2026-02-23T08:23:42.033 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378632' 2026-02-23T08:23:42.033 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 81604378632 2026-02-23T08:23:42.033 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:23:42.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378632 -lt 81604378632 2026-02-23T08:23:42.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:42.244 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116998 2026-02-23T08:23:42.244 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:42.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:23:42.246 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116998 2026-02-23T08:23:42.246 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:42.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116998 2026-02-23T08:23:42.248 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-23T08:23:42.248 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964116998 2026-02-23T08:23:42.248 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:23:42.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116998 -lt 115964116998 2026-02-23T08:23:42.477 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:23:42.477 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:42.477 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:42.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:23:42.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:23:42.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:23:42.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:23:42.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:23:42.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:23:42.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:23:42.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:23:42.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:23:42.964 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:23:42.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:42.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:43.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:23:43.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:23:43.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:23:43.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:23:43.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:23:43.563 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:23:43.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-1-10 2026-02-23T08:23:43.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:23:43.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:23:43.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-1-10 2026-02-23T08:23:43.574 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:23:43.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:23:43.575 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-1-10 td/test-erasure-eio/COPY 2026-02-23T08:23:43.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:23:43.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:23:43.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-23T08:23:43.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-23T08:23:43.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74343-1-10 td/test-erasure-eio 2 10 set 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-1-10 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:23:43.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-1-10 2026-02-23T08:23:43.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:23:43.600 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-1-10 2026-02-23T08:23:43.600 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-1-10 2026-02-23T08:23:43.600 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:23:43.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:23:43.812 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:43.812 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:23:43.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:23:43.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:23:43.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:23:43.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-23T08:23:43.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:23:44.123 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:23:44.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:23:44.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-23T08:23:44.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-23T08:23:44.133 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:23:44.133 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:23:44.133 INFO:tasks.workunit.client.0.trial186.stderr:10 bytes copied, 3.5065e-05 s, 285 kB/s 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:44.134 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:44.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:44.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:44.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:44.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:44.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:44.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:23:44.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:23:44.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-23T08:23:44.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:23:44.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-23T08:23:44.444 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74343-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:44.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-23T08:23:44.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:44.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:23:44.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:23:44.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:23:44.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:23:44.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:44.746 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:23:44.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:23:44.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:23:44.748 INFO:tasks.workunit.client.0.trial186.stderr:start osd.2 2026-02-23T08:23:44.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:44.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:23:44.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:23:44.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:23:44.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:23:44.750 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:23:44.770 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:44.763+0000 7f60f24f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:44.771 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:44.763+0000 7f60f24f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:44.772 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:44.763+0000 7f60f24f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:44.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:23:45.126 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:45.119+0000 7f60f24f2a40 -1 Falling back to public interface 2026-02-23T08:23:45.184 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:45.736 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:45.727+0000 7f60f24f2a40 -1 osd.2 46 log_to_monitors true 2026-02-23T08:23:46.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:46.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:46.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:46.186 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:46.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:46.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:23:46.407 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 up in weight 1 up_from 50 up_thru 38 down_at 47 last_clean_interval [19,46) [v2:127.0.0.1:6818/1146407569,v1:127.0.0.1:6819/1146407569] [v2:127.0.0.1:6820/1146407569,v1:127.0.0.1:6821/1146407569] exists,up c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:23:46.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:46.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:46.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:46.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:23:46.408 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:23:46.408 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:23:46.408 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:23:46.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:23:46.409 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:23:46.409 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:23:46.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:23:46.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:23:46.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:23:46.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:23:46.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:23:46.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:23:46.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:23:46.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:23:46.515 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:23:46.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:23:46.733 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:46.733 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:23:46.733 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:23:46.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:23:46.733 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:46.733 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:23:46.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-23T08:23:46.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-23T08:23:46.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788' 2026-02-23T08:23:46.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:46.812 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:23:46.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659142 2026-02-23T08:23:46.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659142 2026-02-23T08:23:46.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-176093659142' 2026-02-23T08:23:46.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:46.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:23:46.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-23T08:23:46.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-23T08:23:46.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-176093659142 2-214748364803' 2026-02-23T08:23:46.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:46.971 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:23:47.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117002 2026-02-23T08:23:47.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117002 2026-02-23T08:23:47.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-176093659142 2-214748364803 3-115964117002' 2026-02-23T08:23:47.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:47.052 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803788 2026-02-23T08:23:47.052 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:47.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:23:47.054 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-23T08:23:47.054 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:47.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803788 2026-02-23T08:23:47.055 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-23T08:23:47.055 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803788 2026-02-23T08:23:47.056 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:47.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803788 2026-02-23T08:23:47.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:23:48.284 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:23:48.285 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:48.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-23T08:23:48.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:48.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-176093659142 2026-02-23T08:23:48.510 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:48.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:23:48.512 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-176093659142 2026-02-23T08:23:48.512 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:48.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659142 2026-02-23T08:23:48.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 176093659142' 2026-02-23T08:23:48.514 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 176093659142 2026-02-23T08:23:48.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:23:48.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659142 -lt 176093659142 2026-02-23T08:23:48.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:48.739 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-214748364803 2026-02-23T08:23:48.740 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:48.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:23:48.741 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-214748364803 2026-02-23T08:23:48.742 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:48.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364803 2026-02-23T08:23:48.743 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 214748364803' 2026-02-23T08:23:48.743 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 214748364803 2026-02-23T08:23:48.744 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:23:48.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364803 -lt 214748364803 2026-02-23T08:23:48.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:48.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117002 2026-02-23T08:23:48.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:48.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:23:48.971 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117002 2026-02-23T08:23:48.972 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:48.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117002 2026-02-23T08:23:48.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117002' 2026-02-23T08:23:48.973 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117002 2026-02-23T08:23:48.973 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:23:49.195 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117002 -lt 115964117002 2026-02-23T08:23:49.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:23:49.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:49.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:49.484 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:23:49.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:23:49.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:23:49.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:23:49.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:23:49.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:23:49.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:23:49.485 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:23:49.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:23:49.698 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:23:49.698 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:49.699 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:49.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:23:49.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:23:49.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:23:49.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:23:49.990 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:23:50.308 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:23:50.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-23T08:23:50.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-1-10 2026-02-23T08:23:50.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:23:50.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:23:50.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-1-10 2026-02-23T08:23:50.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:23:50.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:23:50.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-1-10 td/test-erasure-eio/COPY 2026-02-23T08:23:50.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:23:50.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:23:50.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:23:50.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74343-1-0 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74343-1-0 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:23:50.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74343-1-0 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:23:50.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74343-1-0 td/test-erasure-eio/ORIGINAL 2026-02-23T08:23:50.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74343-1-0 td/test-erasure-eio 1 0 set 2026-02-23T08:23:50.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-1-0 2026-02-23T08:23:50.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:23:50.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:23:50.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:23:50.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-1-0 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-1-0 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-1-0 2026-02-23T08:23:50.374 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:23:50.587 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:23:50.587 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:50.587 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:23:50.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:23:50.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:23:50.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:23:50.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-23T08:23:50.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:23:50.878 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:23:50.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:23:50.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-23T08:23:50.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-23T08:23:50.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:50.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:50.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:51.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:51.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:51.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:23:51.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:23:51.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:23:51.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:23:51.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:51.200 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:51.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:51.502 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:51.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:23:51.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:23:51.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:23:51.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:23:51.504 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:23:51.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:51.506 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:23:51.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:23:51.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:23:51.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:23:51.507 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:23:51.524 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:51.515+0000 7fe22a73ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:51.525 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:51.519+0000 7fe22a73ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:51.526 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:51.519+0000 7fe22a73ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:51.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:23:51.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:51.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:23:51.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:51.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:51.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:51.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:51.726 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:23:51.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:51.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:51.899 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:51.891+0000 7fe22a73ea40 -1 Falling back to public interface 2026-02-23T08:23:51.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:23:52.527 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:52.519+0000 7fe22a73ea40 -1 osd.1 53 log_to_monitors true 2026-02-23T08:23:52.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:23:52.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:52.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:23:52.949 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:52.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:52.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:23:53.170 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [41,53) [v2:127.0.0.1:6810/915024992,v1:127.0.0.1:6811/915024992] [v2:127.0.0.1:6812/915024992,v1:127.0.0.1:6813/915024992] exists,up a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:23:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:23:53.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:23:53.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:23:53.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:23:53.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:23:53.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:23:53.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:23:53.171 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:23:53.172 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:23:53.172 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:23:53.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:23:53.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:23:53.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:23:53.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:23:53.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:23:53.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:23:53.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:23:53.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:23:53.276 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:23:53.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:23:53.493 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:53.493 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:23:53.493 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:23:53.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:23:53.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:53.493 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:23:53.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-23T08:23:53.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-23T08:23:53.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-23T08:23:53.576 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:53.576 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:23:53.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-23T08:23:53.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-23T08:23:53.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-253403070467' 2026-02-23T08:23:53.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:53.655 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:23:53.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364806 2026-02-23T08:23:53.735 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364806 2026-02-23T08:23:53.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-253403070467 2-214748364806' 2026-02-23T08:23:53.736 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:23:53.736 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:23:53.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117005 2026-02-23T08:23:53.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117005 2026-02-23T08:23:53.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-253403070467 2-214748364806 3-115964117005' 2026-02-23T08:23:53.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:53.815 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-23T08:23:53.815 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:53.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:23:53.817 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-23T08:23:53.817 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-23T08:23:53.818 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-23T08:23:53.818 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803791 2026-02-23T08:23:53.819 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:54.038 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-23T08:23:54.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:23:55.039 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:23:55.040 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:55.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-23T08:23:55.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:23:56.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-23T08:23:56.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:23:56.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-23T08:23:56.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:56.476 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-253403070467 2026-02-23T08:23:56.476 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:56.477 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:23:56.478 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-253403070467 2026-02-23T08:23:56.478 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:56.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=253403070467 2026-02-23T08:23:56.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 253403070467' 2026-02-23T08:23:56.480 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 253403070467 2026-02-23T08:23:56.480 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:23:56.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-23T08:23:56.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:56.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-214748364806 2026-02-23T08:23:56.708 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:56.709 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:23:56.710 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-214748364806 2026-02-23T08:23:56.710 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:56.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364806 2026-02-23T08:23:56.712 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 214748364806' 2026-02-23T08:23:56.712 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 214748364806 2026-02-23T08:23:56.712 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:23:56.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364806 -lt 214748364806 2026-02-23T08:23:56.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:23:56.931 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117005 2026-02-23T08:23:56.932 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:23:56.933 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:23:56.934 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117005 2026-02-23T08:23:56.934 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:23:56.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117005 2026-02-23T08:23:56.935 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117005' 2026-02-23T08:23:56.935 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117005 2026-02-23T08:23:56.936 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:23:57.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117005 -lt 115964117005 2026-02-23T08:23:57.164 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:23:57.164 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:57.164 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:57.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:23:57.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:23:57.459 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:23:57.460 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:23:57.460 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:23:57.460 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:23:57.460 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:23:57.460 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:23:57.675 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:23:57.676 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:23:57.676 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:23:57.676 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:23:57.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:23:57.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:23:57.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:23:57.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:23:57.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:23:58.285 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:23:58.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-1-0 2026-02-23T08:23:58.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:23:58.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:23:58.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-1-0 2026-02-23T08:23:58.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:23:58.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:23:58.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-1-0 td/test-erasure-eio/COPY 2026-02-23T08:23:58.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:23:58.325 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:23:58.327 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-23T08:23:58.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74343-1-0 td/test-erasure-eio 2 0 set 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-1-0 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:23:58.329 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-23T08:23:58.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:23:58.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-1-0 2026-02-23T08:23:58.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:23:58.330 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-1-0 2026-02-23T08:23:58.330 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-1-0 2026-02-23T08:23:58.330 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:23:58.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:23:58.551 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:23:58.551 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:23:58.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:23:58.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:23:58.551 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:23:58.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-23T08:23:58.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:23:58.838 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:23:58.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-23T08:23:58.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-23T08:23:58.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-23T08:23:58.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:58.851 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:23:58.852 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:23:58.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:23:58.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:23:58.853 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-23T08:23:59.161 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74343-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:23:59.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-23T08:23:59.454 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:23:59.455 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:23:59.456 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:23:59.456 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:23:59.457 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:23:59.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:23:59.458 INFO:tasks.workunit.client.0.trial186.stderr:start osd.2 2026-02-23T08:23:59.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:23:59.458 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:23:59.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:23:59.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:23:59.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:23:59.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:23:59.472 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:59.463+0000 7febd4b38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:59.473 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:59.467+0000 7febd4b38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:59.474 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:59.467+0000 7febd4b38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:23:59.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:23:59.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:23:59.851 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:23:59.843+0000 7febd4b38a40 -1 Falling back to public interface 2026-02-23T08:23:59.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:00.444 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:00.435+0000 7febd4b38a40 -1 osd.2 63 log_to_monitors true 2026-02-23T08:24:00.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:00.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:00.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:00.894 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:00.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:00.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:24:01.108 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 up in weight 1 up_from 68 up_thru 56 down_at 65 last_clean_interval [50,63) [v2:127.0.0.1:6818/714210949,v1:127.0.0.1:6819/714210949] [v2:127.0.0.1:6820/714210949,v1:127.0.0.1:6821/714210949] exists,up c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:24:01.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:24:01.110 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:24:01.110 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:24:01.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:24:01.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:24:01.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:24:01.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:24:01.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:24:01.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:24:01.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:24:01.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:24:01.213 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:24:01.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:24:01.426 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:01.426 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:24:01.426 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:24:01.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:24:01.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:01.426 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:24:01.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803795 2026-02-23T08:24:01.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803795 2026-02-23T08:24:01.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795' 2026-02-23T08:24:01.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:01.508 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:24:01.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070470 2026-02-23T08:24:01.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070470 2026-02-23T08:24:01.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-253403070470' 2026-02-23T08:24:01.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:01.589 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:24:01.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-23T08:24:01.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-23T08:24:01.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-253403070470 2-292057776131' 2026-02-23T08:24:01.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:01.672 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:24:01.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117008 2026-02-23T08:24:01.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117008 2026-02-23T08:24:01.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-253403070470 2-292057776131 3-115964117008' 2026-02-23T08:24:01.749 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:01.749 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803795 2026-02-23T08:24:01.750 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:01.751 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:24:01.752 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803795 2026-02-23T08:24:01.752 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:01.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803795 2026-02-23T08:24:01.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-23T08:24:01.754 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803795 2026-02-23T08:24:01.754 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:01.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803795 2026-02-23T08:24:01.978 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:24:02.979 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:24:02.980 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:03.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803795 2026-02-23T08:24:03.196 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:24:04.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-23T08:24:04.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:04.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803795 2026-02-23T08:24:04.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:04.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-253403070470 2026-02-23T08:24:04.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:04.418 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:24:04.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-253403070470 2026-02-23T08:24:04.419 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:04.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=253403070470 2026-02-23T08:24:04.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 253403070470' 2026-02-23T08:24:04.420 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 253403070470 2026-02-23T08:24:04.421 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:24:04.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070471 -lt 253403070470 2026-02-23T08:24:04.648 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:04.648 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-292057776131 2026-02-23T08:24:04.648 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:04.650 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:24:04.650 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776131 2026-02-23T08:24:04.650 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:04.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776131 2026-02-23T08:24:04.652 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 292057776131' 2026-02-23T08:24:04.652 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 292057776131 2026-02-23T08:24:04.652 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:24:04.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-23T08:24:04.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:04.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117008 2026-02-23T08:24:04.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:04.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:24:04.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117008 2026-02-23T08:24:04.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:04.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117008 2026-02-23T08:24:04.884 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117008' 2026-02-23T08:24:04.884 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117008 2026-02-23T08:24:04.885 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:24:05.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117009 -lt 115964117008 2026-02-23T08:24:05.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:24:05.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:05.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:05.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:24:05.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:05.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:05.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:05.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:05.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:05.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:05.414 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:05.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:24:05.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:05.634 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:05.634 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:05.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:24:05.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:24:05.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:24:05.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:24:05.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:24:06.281 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-1-0 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-1-0 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:24:06.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-1-0 td/test-erasure-eio/COPY 2026-02-23T08:24:06.315 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:24:06.316 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:24:06.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:24:06.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-23T08:24:06.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-74343-1-256 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-74343-1-256 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:24:06.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-74343-1-256 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:24:06.322 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-74343-1-256 td/test-erasure-eio/ORIGINAL 2026-02-23T08:24:06.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-74343-1-256 td/test-erasure-eio 1 256 add 2026-02-23T08:24:06.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-1-256 2026-02-23T08:24:06.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:24:06.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:24:06.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-1-256 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:24:06.346 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-1-256 2026-02-23T08:24:06.347 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-1-256 2026-02-23T08:24:06.347 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:24:06.563 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:24:06.563 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:06.564 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:24:06.564 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:24:06.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:24:06.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:24:06.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-23T08:24:06.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:24:06.853 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:24:06.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:24:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:24:06.867 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:06.867 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:06.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:24:07.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:07.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:24:07.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:07.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:07.350 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:24:07.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:24:07.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:24:07.352 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:24:07.352 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:07.353 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:24:07.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:24:07.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:24:07.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:24:07.354 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:24:07.370 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:07.363+0000 7f146ba80a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:07.372 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:07.363+0000 7f146ba80a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:07.372 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:07.363+0000 7f146ba80a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:07.572 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:24:07.573 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:07.573 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:24:07.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:07.835 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:07.827+0000 7f146ba80a40 -1 Falling back to public interface 2026-02-23T08:24:08.422 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:08.415+0000 7f146ba80a40 -1 osd.1 73 log_to_monitors true 2026-02-23T08:24:08.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:08.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:08.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:08.785 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:08.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:08.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,73) [v2:127.0.0.1:6810/581915369,v1:127.0.0.1:6811/581915369] [v2:127.0.0.1:6812/581915369,v1:127.0.0.1:6813/581915369] exists,up a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:24:08.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:24:08.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:24:08.998 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:24:08.999 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:24:08.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:24:08.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:24:08.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:24:09.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:24:09.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:24:09.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:24:09.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:24:09.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:24:09.104 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:24:09.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:24:09.320 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:09.320 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:24:09.320 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:24:09.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:24:09.320 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:09.321 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:24:09.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803798 2026-02-23T08:24:09.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803798 2026-02-23T08:24:09.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798' 2026-02-23T08:24:09.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:09.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:24:09.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481795 2026-02-23T08:24:09.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481795 2026-02-23T08:24:09.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-330712481795' 2026-02-23T08:24:09.479 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:09.479 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:24:09.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776134 2026-02-23T08:24:09.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776134 2026-02-23T08:24:09.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-330712481795 2-292057776134' 2026-02-23T08:24:09.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:09.560 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:24:09.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117012 2026-02-23T08:24:09.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117012 2026-02-23T08:24:09.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-330712481795 2-292057776134 3-115964117012' 2026-02-23T08:24:09.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:09.643 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803798 2026-02-23T08:24:09.643 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:09.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:24:09.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803798 2026-02-23T08:24:09.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:09.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803798 2026-02-23T08:24:09.646 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803798' 2026-02-23T08:24:09.646 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803798 2026-02-23T08:24:09.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:09.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803796 -lt 25769803798 2026-02-23T08:24:09.866 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:24:10.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:24:10.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:11.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803796 -lt 25769803798 2026-02-23T08:24:11.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:24:12.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-23T08:24:12.088 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:12.300 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803798 -lt 25769803798 2026-02-23T08:24:12.301 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:12.301 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-330712481795 2026-02-23T08:24:12.301 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:12.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:24:12.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-330712481795 2026-02-23T08:24:12.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:12.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=330712481795 2026-02-23T08:24:12.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 330712481795' 2026-02-23T08:24:12.305 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 330712481795 2026-02-23T08:24:12.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:24:12.523 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-23T08:24:12.524 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:12.524 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-292057776134 2026-02-23T08:24:12.524 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:12.526 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:24:12.526 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776134 2026-02-23T08:24:12.526 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:12.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776134 2026-02-23T08:24:12.528 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 292057776134' 2026-02-23T08:24:12.528 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 292057776134 2026-02-23T08:24:12.528 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:24:12.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776135 -lt 292057776134 2026-02-23T08:24:12.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:12.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117012 2026-02-23T08:24:12.759 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:12.760 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:24:12.760 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117012 2026-02-23T08:24:12.761 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:12.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117012 2026-02-23T08:24:12.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117012' 2026-02-23T08:24:12.762 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117012 2026-02-23T08:24:12.763 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:24:12.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117012 -lt 115964117012 2026-02-23T08:24:12.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:24:12.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:12.985 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:13.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:13.495 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:24:13.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:13.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:13.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:13.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:24:13.787 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:24:13.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:24:13.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-23T08:24:13.790 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:24:13.790 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:24:13.790 INFO:tasks.workunit.client.0.trial186.stderr:256 bytes copied, 2.0297e-05 s, 12.6 MB/s 2026-02-23T08:24:13.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:13.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:24:13.790 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:13.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:13.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:14.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:14.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:14.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:24:14.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:24:14.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:24:14.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:24:14.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:24:14.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:14.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:14.363 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:14.363 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:14.363 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:14.363 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:14.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:14.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:14.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:24:14.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:24:14.365 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:24:14.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:14.365 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:24:14.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:24:14.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:24:14.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:24:14.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:24:14.382 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:14.375+0000 7f4beb97aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:14.384 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:14.375+0000 7f4beb97aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:14.385 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:14.375+0000 7f4beb97aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:14.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:14.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:24:14.778 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:14.771+0000 7f4beb97aa40 -1 Falling back to public interface 2026-02-23T08:24:14.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:15.417 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:15.407+0000 7f4beb97aa40 -1 osd.1 78 log_to_monitors true 2026-02-23T08:24:15.804 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:15.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:15.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:15.805 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:15.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:15.805 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:24:16.024 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [77,78) [v2:127.0.0.1:6810/1421001037,v1:127.0.0.1:6811/1421001037] [v2:127.0.0.1:6812/1421001037,v1:127.0.0.1:6813/1421001037] exists,up a0e80d37-0449-4cd6-ad23-f2406a6c2017 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:24:16.025 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:24:16.026 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:24:16.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:24:16.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:24:16.026 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:24:16.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:24:16.128 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:24:16.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:24:16.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:24:16.129 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:24:16.129 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:24:16.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:24:16.348 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:16.348 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:24:16.348 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:24:16.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:24:16.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:16.349 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:24:16.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803802 2026-02-23T08:24:16.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803802 2026-02-23T08:24:16.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802' 2026-02-23T08:24:16.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:16.429 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:24:16.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318275 2026-02-23T08:24:16.510 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318275 2026-02-23T08:24:16.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-352187318275' 2026-02-23T08:24:16.511 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:16.511 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:24:16.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776138 2026-02-23T08:24:16.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776138 2026-02-23T08:24:16.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-352187318275 2-292057776138' 2026-02-23T08:24:16.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:16.591 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:24:16.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117015 2026-02-23T08:24:16.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117015 2026-02-23T08:24:16.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-352187318275 2-292057776138 3-115964117015' 2026-02-23T08:24:16.671 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:16.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803802 2026-02-23T08:24:16.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:16.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:24:16.673 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803802 2026-02-23T08:24:16.673 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:16.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803802 2026-02-23T08:24:16.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-23T08:24:16.674 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803802 2026-02-23T08:24:16.675 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:16.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803799 -lt 25769803802 2026-02-23T08:24:16.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:24:17.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:24:17.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:18.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-23T08:24:18.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:18.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-352187318275 2026-02-23T08:24:18.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:18.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:24:18.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-352187318275 2026-02-23T08:24:18.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:18.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=352187318275 2026-02-23T08:24:18.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 352187318275' 2026-02-23T08:24:18.113 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 352187318275 2026-02-23T08:24:18.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:24:18.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 352187318275 -lt 352187318275 2026-02-23T08:24:18.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:18.333 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-292057776138 2026-02-23T08:24:18.333 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:18.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:24:18.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776138 2026-02-23T08:24:18.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:18.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776138 2026-02-23T08:24:18.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 292057776138' 2026-02-23T08:24:18.337 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 292057776138 2026-02-23T08:24:18.337 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:24:18.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776138 -lt 292057776138 2026-02-23T08:24:18.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:18.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117015 2026-02-23T08:24:18.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:18.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:24:18.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117015 2026-02-23T08:24:18.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:18.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117015 2026-02-23T08:24:18.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117015' 2026-02-23T08:24:18.560 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117015 2026-02-23T08:24:18.561 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:24:18.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117016 -lt 115964117015 2026-02-23T08:24:18.789 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:24:18.790 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:18.790 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:19.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:24:19.082 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:19.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:19.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:19.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:19.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:19.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:19.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:19.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:24:19.298 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:19.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:19.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:19.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:24:19.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:24:19.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:24:19.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:24:19.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:24:19.918 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:24:19.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-1-256 2026-02-23T08:24:19.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:24:19.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:24:19.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-1-256 2026-02-23T08:24:19.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:24:19.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:24:19.928 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-1-256 td/test-erasure-eio/COPY 2026-02-23T08:24:19.955 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:24:19.956 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:24:19.958 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-74343-1-256 td/test-erasure-eio 2 256 add 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-74343-1-256 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-23T08:24:19.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-23T08:24:19.960 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-74343-1-256 2026-02-23T08:24:19.960 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:24:19.960 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-74343-1-256 2026-02-23T08:24:19.960 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-74343-1-256 2026-02-23T08:24:19.961 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:24:20.182 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:24:20.183 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:20.183 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:24:20.183 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:24:20.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-23T08:24:20.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-23T08:24:20.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-23T08:24:20.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-23T08:24:20.491 INFO:tasks.workunit.client.0.trial186.stderr:noout is set 2026-02-23T08:24:20.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-23T08:24:20.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:20.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:20.503 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:20.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:20.504 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:20.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:20.810 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:20.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:24:20.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:24:20.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-23T08:24:20.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:24:20.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-23T08:24:20.811 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74343-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:20.997 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:20.998 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:20.999 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:20.999 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:24:21.000 INFO:tasks.workunit.client.0.trial186.stderr:start osd.2 2026-02-23T08:24:21.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:21.001 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:24:21.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:24:21.001 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:24:21.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:24:21.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:24:21.018 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:21.011+0000 7f4992537a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:21.019 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:21.011+0000 7f4992537a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:21.020 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:21.011+0000 7f4992537a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:21.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:24:21.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:21.475 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:21.467+0000 7f4992537a40 -1 Falling back to public interface 2026-02-23T08:24:22.149 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:22.139+0000 7f4992537a40 -1 osd.2 85 log_to_monitors true 2026-02-23T08:24:22.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:22.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:22.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:22.427 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:22.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:22.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:24:22.642 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 up in weight 1 up_from 91 up_thru 79 down_at 88 last_clean_interval [68,85) [v2:127.0.0.1:6818/1956866387,v1:127.0.0.1:6819/1956866387] [v2:127.0.0.1:6820/1956866387,v1:127.0.0.1:6821/1956866387] exists,up c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:24:22.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:22.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:22.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:22.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:24:22.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:24:22.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:24:22.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:24:22.643 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:24:22.644 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:24:22.644 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:24:22.644 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:24:22.644 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:24:22.644 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:24:22.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:24:22.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:24:22.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:24:22.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:24:22.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:24:22.747 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:24:22.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:24:22.969 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:22.969 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:24:22.969 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:24:22.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:24:22.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:22.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:24:23.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803805 2026-02-23T08:24:23.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803805 2026-02-23T08:24:23.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805' 2026-02-23T08:24:23.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:23.051 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:24:23.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318278 2026-02-23T08:24:23.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318278 2026-02-23T08:24:23.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-352187318278' 2026-02-23T08:24:23.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:23.130 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:24:23.211 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=390842023939 2026-02-23T08:24:23.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 390842023939 2026-02-23T08:24:23.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-352187318278 2-390842023939' 2026-02-23T08:24:23.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:23.212 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:24:23.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117019 2026-02-23T08:24:23.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117019 2026-02-23T08:24:23.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-352187318278 2-390842023939 3-115964117019' 2026-02-23T08:24:23.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:23.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803805 2026-02-23T08:24:23.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:23.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:24:23.292 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803805 2026-02-23T08:24:23.292 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:23.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803805 2026-02-23T08:24:23.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803805' 2026-02-23T08:24:23.294 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803805 2026-02-23T08:24:23.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:23.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803805 2026-02-23T08:24:23.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:23.518 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-352187318278 2026-02-23T08:24:23.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:23.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:24:23.521 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-352187318278 2026-02-23T08:24:23.521 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:23.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=352187318278 2026-02-23T08:24:23.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 352187318278' 2026-02-23T08:24:23.522 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 352187318278 2026-02-23T08:24:23.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:24:23.737 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 352187318278 -lt 352187318278 2026-02-23T08:24:23.737 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:23.738 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-390842023939 2026-02-23T08:24:23.738 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:23.739 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:24:23.740 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-390842023939 2026-02-23T08:24:23.740 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:23.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=390842023939 2026-02-23T08:24:23.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 390842023939' 2026-02-23T08:24:23.742 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 390842023939 2026-02-23T08:24:23.742 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:24:23.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 390842023939 -lt 390842023939 2026-02-23T08:24:23.965 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:23.966 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117019 2026-02-23T08:24:23.966 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:23.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:24:23.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117019 2026-02-23T08:24:23.967 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:23.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117019 2026-02-23T08:24:23.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117019' 2026-02-23T08:24:23.969 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117019 2026-02-23T08:24:23.969 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:24:24.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117019 -lt 115964117019 2026-02-23T08:24:24.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:24:24.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:24.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:24.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:24:24.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:24.489 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:24.489 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:24.489 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:24.489 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:24.490 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:24.490 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:24.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-23T08:24:24.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:24.704 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:24.704 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:24.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-23T08:24:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' -1 2026-02-23T08:24:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-23T08:24:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=3 2026-02-23T08:24:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-23T08:24:24.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-23T08:24:25.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-23T08:24:25.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:25.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:25.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:25.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:25.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:25.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:25.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:25.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-23T08:24:25.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:25.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:25.306 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:25.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-23T08:24:25.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-23T08:24:25.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-23T08:24:25.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-23T08:24:25.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-23T08:24:25.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-23T08:24:25.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-23T08:24:25.589 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-23T08:24:25.589 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-23T08:24:25.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-23T08:24:25.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-23T08:24:25.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-23T08:24:25.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-23T08:24:25.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:26.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:26.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=3 2026-02-23T08:24:26.297 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:26.297 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:26.298 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 = 5 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 3 '!=' 3 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-23T08:24:26.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-23T08:24:26.589 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-23T08:24:26.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-23T08:24:26.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-23T08:24:26.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-23T08:24:26.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-23T08:24:26.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-23T08:24:27.284 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-23T08:24:27.284 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:27.285 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:27.285 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:27.285 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:27.285 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:27.286 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:27.286 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:27.502 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:24:27.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:27.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:27.503 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:27.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:24:27.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:24:27.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:24:27.798 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-23T08:24:27.800 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records in 2026-02-23T08:24:27.800 INFO:tasks.workunit.client.0.trial186.stderr:1+0 records out 2026-02-23T08:24:27.800 INFO:tasks.workunit.client.0.trial186.stderr:256 bytes copied, 2.3173e-05 s, 11.0 MB/s 2026-02-23T08:24:27.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:27.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:27.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:27.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:27.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-23T08:24:28.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-74343-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:28.356 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:28.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:24:28.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:24:28.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:24:28.360 INFO:tasks.workunit.client.0.trial186.stderr:start osd.2 2026-02-23T08:24:28.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:28.360 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:24:28.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:24:28.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:24:28.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:24:28.361 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:24:28.373 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:28.363+0000 7f62ca923a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:28.375 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:28.367+0000 7f62ca923a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:28.376 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:28.367+0000 7f62ca923a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:28.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:28.578 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:24:28.797 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:28.851 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:28.843+0000 7f62ca923a40 -1 Falling back to public interface 2026-02-23T08:24:29.443 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:29.435+0000 7f62ca923a40 -1 osd.2 92 log_to_monitors true 2026-02-23T08:24:29.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:29.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:29.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:29.799 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:29.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:29.799 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:24:30.015 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 up in weight 1 up_from 96 up_thru 79 down_at 93 last_clean_interval [91,92) [v2:127.0.0.1:6818/1097843589,v1:127.0.0.1:6819/1097843589] [v2:127.0.0.1:6820/1097843589,v1:127.0.0.1:6821/1097843589] exists,up c7aee2d9-bf35-49d7-9277-135d1aff6232 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:24:30.016 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:24:30.017 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:24:30.017 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:24:30.018 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:24:30.018 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:24:30.018 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:24:30.125 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:24:30.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:24:30.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:24:30.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:24:30.126 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:24:30.126 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:24:30.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:24:30.345 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:30.345 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:24:30.345 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:24:30.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:24:30.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:30.346 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:24:30.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803808 2026-02-23T08:24:30.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803808 2026-02-23T08:24:30.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808' 2026-02-23T08:24:30.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:30.428 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:24:30.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318282 2026-02-23T08:24:30.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318282 2026-02-23T08:24:30.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-352187318282' 2026-02-23T08:24:30.506 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:30.506 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:24:30.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860419 2026-02-23T08:24:30.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860419 2026-02-23T08:24:30.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-352187318282 2-412316860419' 2026-02-23T08:24:30.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:30.588 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:24:30.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117022 2026-02-23T08:24:30.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117022 2026-02-23T08:24:30.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-352187318282 2-412316860419 3-115964117022' 2026-02-23T08:24:30.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:30.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803808 2026-02-23T08:24:30.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:30.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:24:30.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803808 2026-02-23T08:24:30.669 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:30.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803808 2026-02-23T08:24:30.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803808' 2026-02-23T08:24:30.670 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803808 2026-02-23T08:24:30.671 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:30.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803806 -lt 25769803808 2026-02-23T08:24:30.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:24:31.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:24:31.893 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:32.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803808 -lt 25769803808 2026-02-23T08:24:32.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:32.106 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-352187318282 2026-02-23T08:24:32.106 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:32.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:24:32.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-352187318282 2026-02-23T08:24:32.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:32.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=352187318282 2026-02-23T08:24:32.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 352187318282' 2026-02-23T08:24:32.110 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 352187318282 2026-02-23T08:24:32.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:24:32.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 352187318282 -lt 352187318282 2026-02-23T08:24:32.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:32.331 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-412316860419 2026-02-23T08:24:32.331 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:32.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:24:32.333 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-412316860419 2026-02-23T08:24:32.333 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:32.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=412316860419 2026-02-23T08:24:32.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 412316860419' 2026-02-23T08:24:32.335 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 412316860419 2026-02-23T08:24:32.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:24:32.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-23T08:24:32.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:32.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117022 2026-02-23T08:24:32.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:32.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:24:32.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117022 2026-02-23T08:24:32.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:32.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117022 2026-02-23T08:24:32.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117022' 2026-02-23T08:24:32.552 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117022 2026-02-23T08:24:32.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:24:32.770 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117022 -lt 115964117022 2026-02-23T08:24:32.770 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:24:32.770 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:32.771 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:33.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:24:33.062 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:33.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:33.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:33.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:33.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:33.062 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:33.063 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:33.276 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:24:33.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:33.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:33.277 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:33.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:24:33.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:24:33.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:24:33.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-23T08:24:33.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-23T08:24:33.882 INFO:tasks.workunit.client.0.trial186.stderr:noout is unset 2026-02-23T08:24:33.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-23T08:24:33.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-74343-1-256 2026-02-23T08:24:33.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:24:33.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:24:33.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-74343-1-256 2026-02-23T08:24:33.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:24:33.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:24:33.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-74343-1-256 td/test-erasure-eio/COPY 2026-02-23T08:24:33.916 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:24:33.917 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:24:33.918 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:24:33.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:24:33.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:24:33.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:24:34.141 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:24:34.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:24:34.432 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:24:34.442 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:34.443 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:34.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:34.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:24:34.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:24:34.556 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:24:34.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:24:34.558 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:24:34.558 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:24:34.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:34.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:24:34.560 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:24:34.560 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:34.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:24:34.562 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:24:34.563 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:24:34.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:24:34.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:34.586 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:34.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:24:34.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:24:34.589 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:34.589 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:34.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:34.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:34.589 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:34.591 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:34.591 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:24:34.592 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:24:34.592 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:24:34.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:24:34.594 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:24:34.594 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:24:34.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:34.595 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:24:34.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:24:34.596 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:34.597 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:24:34.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:24:34.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:24:34.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:24:34.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:34.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:34.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:24:34.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:24:34.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:24:34.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:24:34.602 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:24:34.603 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:34.603 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:34.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:24:34.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:24:34.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:24:34.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:24:34.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:24:34.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:24:34.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:24:34.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:24:34.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:24:34.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:24:34.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:24:34.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:24:34.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:24:34.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:34.627 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:34.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:34.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:34.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:34.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:34.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:24:34.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:24:34.658 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:24:34.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:24:34.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:24:34.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:24:34.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:24:34.659 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:24:34.659 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:24:34.659 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:24:34.661 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:34.661 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:34.661 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:34.661 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:24:34.661 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:24:34.661 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:24:34.719 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:24:34.721 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:34.721 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:34.721 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:34.721 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:24:34.721 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:24:34.721 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:24:34.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:24:34.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:24:34.778 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:24:34.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:24:34.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:24:34.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:24:34.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:24:34.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:24:34.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:34.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:34.892 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:34.892 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:34.893 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:34.893 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:34.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:24:34.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:24:34.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:24:34.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:24:35.020 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:24:35.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:24:36.030 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:24:36.030 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:24:36.030 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:24:36.030 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:36.030 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:36.030 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:36.031 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:24:36.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:24:36.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:24:36.083 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:24:36.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:24:36.090 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:24:34.638+0000 7fa0d5b5fec0 0 load: jerasure load: lrc 2026-02-23T08:24:36.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-23T08:24:36.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-23T08:24:36.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-23T08:24:36.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-23T08:24:36.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:24:36.092 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-23T08:24:36.092 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-23T08:24:36.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:24:36.096 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:36.097 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:36.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:24:36.099 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:24:36.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=487ee64a-0554-44df-9a14-18e9936a07bb 2026-02-23T08:24:36.100 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 487ee64a-0554-44df-9a14-18e9936a07bb 2026-02-23T08:24:36.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 487ee64a-0554-44df-9a14-18e9936a07bb' 2026-02-23T08:24:36.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:24:36.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDEDpxpSAsMBhAAUaptaf+KNoi/EinsRbnPiA== 2026-02-23T08:24:36.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDEDpxpSAsMBhAAUaptaf+KNoi/EinsRbnPiA=="}' 2026-02-23T08:24:36.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 487ee64a-0554-44df-9a14-18e9936a07bb -i td/test-erasure-eio/0/new.json 2026-02-23T08:24:36.226 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:24:36.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:24:36.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDEDpxpSAsMBhAAUaptaf+KNoi/EinsRbnPiA== --osd-uuid 487ee64a-0554-44df-9a14-18e9936a07bb 2026-02-23T08:24:36.252 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:36.242+0000 7fd45fe92a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:36.254 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:36.246+0000 7fd45fe92a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:36.255 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:36.246+0000 7fd45fe92a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:36.256 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:36.246+0000 7fd45fe92a40 -1 bdev(0x55908bc50400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:24:36.256 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:36.246+0000 7fd45fe92a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:24:36.933 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:24:36.933 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:24:36.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:24:36.934 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:24:36.934 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:24:37.085 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:24:37.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:24:37.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:24:37.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:24:37.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:24:37.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:37.101 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:37.090+0000 7f400398ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:37.102 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:37.094+0000 7f400398ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:37.103 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:37.094+0000 7f400398ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:37.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:24:37.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:37.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:24:37.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:37.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:37.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:37.199 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:24:37.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:37.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:37.199 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:24:37.412 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:37.487 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:37.478+0000 7f400398ea40 -1 Falling back to public interface 2026-02-23T08:24:38.031 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:38.022+0000 7f400398ea40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:24:38.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:38.413 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:38.413 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:24:38.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:38.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:38.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:24:38.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:39.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:39.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:39.632 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:24:39.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:24:39.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:39.633 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:24:39.837 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1282446359,v1:127.0.0.1:6803/1282446359] [v2:127.0.0.1:6804/1282446359,v1:127.0.0.1:6805/1282446359] exists,up 487ee64a-0554-44df-9a14-18e9936a07bb 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:39.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:39.839 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:24:39.840 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:24:39.841 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:24:39.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cc9b6a76-501e-4b4f-98d0-635f8e9b2902 2026-02-23T08:24:39.842 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 cc9b6a76-501e-4b4f-98d0-635f8e9b2902 2026-02-23T08:24:39.842 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 cc9b6a76-501e-4b4f-98d0-635f8e9b2902' 2026-02-23T08:24:39.842 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:24:39.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDHDpxpB2eUMhAAPyxCxx1kYqqr14Mp+tgMqw== 2026-02-23T08:24:39.855 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDHDpxpB2eUMhAAPyxCxx1kYqqr14Mp+tgMqw=="}' 2026-02-23T08:24:39.856 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cc9b6a76-501e-4b4f-98d0-635f8e9b2902 -i td/test-erasure-eio/1/new.json 2026-02-23T08:24:40.073 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:24:40.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:24:40.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDHDpxpB2eUMhAAPyxCxx1kYqqr14Mp+tgMqw== --osd-uuid cc9b6a76-501e-4b4f-98d0-635f8e9b2902 2026-02-23T08:24:40.100 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:40.090+0000 7f46bab75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:40.101 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:40.090+0000 7f46bab75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:40.102 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:40.094+0000 7f46bab75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:40.103 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:40.094+0000 7f46bab75a40 -1 bdev(0x5586bbe66400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:24:40.103 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:40.094+0000 7f46bab75a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:24:40.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:24:40.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:24:40.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:24:40.727 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:24:40.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:24:41.025 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:24:41.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:24:41.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:41.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:24:41.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:24:41.026 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:24:41.040 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:41.030+0000 7f5537a58a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:41.041 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:41.034+0000 7f5537a58a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:41.042 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:41.034+0000 7f5537a58a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:41.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:24:41.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:41.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:24:41.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:41.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:41.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:41.245 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:24:41.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:41.245 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:41.246 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:24:41.427 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:41.418+0000 7f5537a58a40 -1 Falling back to public interface 2026-02-23T08:24:41.456 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:42.089 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:42.078+0000 7f5537a58a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:24:42.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:42.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:42.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:42.459 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:24:42.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:42.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:24:42.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:43.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:43.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:43.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:24:43.673 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:24:43.673 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:43.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:24:43.885 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/75955170,v1:127.0.0.1:6811/75955170] [v2:127.0.0.1:6812/75955170,v1:127.0.0.1:6813/75955170] exists,up cc9b6a76-501e-4b4f-98d0-635f8e9b2902 2026-02-23T08:24:43.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:43.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:43.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:43.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:24:43.889 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:24:43.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:43.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:24:43.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:24:43.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=24b76a58-3a69-44b8-b0bb-d061816d6b9b 2026-02-23T08:24:43.893 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 24b76a58-3a69-44b8-b0bb-d061816d6b9b 2026-02-23T08:24:43.894 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 24b76a58-3a69-44b8-b0bb-d061816d6b9b' 2026-02-23T08:24:43.894 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:24:43.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDLDpxp2JZvNRAA5FPVEViBE86DETI+drFLLA== 2026-02-23T08:24:43.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDLDpxp2JZvNRAA5FPVEViBE86DETI+drFLLA=="}' 2026-02-23T08:24:43.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 24b76a58-3a69-44b8-b0bb-d061816d6b9b -i td/test-erasure-eio/2/new.json 2026-02-23T08:24:44.122 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:24:44.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:24:44.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDLDpxp2JZvNRAA5FPVEViBE86DETI+drFLLA== --osd-uuid 24b76a58-3a69-44b8-b0bb-d061816d6b9b 2026-02-23T08:24:44.148 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:44.138+0000 7fa5f1119a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:44.149 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:44.138+0000 7fa5f1119a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:44.151 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:44.142+0000 7fa5f1119a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:44.151 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:44.142+0000 7fa5f1119a40 -1 bdev(0x5569e99ac400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:24:44.152 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:44.142+0000 7fa5f1119a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:24:44.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:24:44.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:24:44.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:24:44.802 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:24:44.802 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:24:45.093 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:24:45.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:24:45.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:45.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:24:45.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:24:45.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:24:45.117 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:45.106+0000 7f30552c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:45.119 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:45.110+0000 7f30552c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:45.120 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:45.110+0000 7f30552c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:45.317 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:24:45.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:45.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:24:45.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:45.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:45.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:45.318 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:45.318 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:24:45.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:45.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:24:45.483 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:45.474+0000 7f30552c6a40 -1 Falling back to public interface 2026-02-23T08:24:45.534 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:46.073 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:46.062+0000 7f30552c6a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:24:46.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:46.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:46.535 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:24:46.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:46.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:46.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:24:46.750 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4099168175,v1:127.0.0.1:6819/4099168175] [v2:127.0.0.1:6820/4099168175,v1:127.0.0.1:6821/4099168175] exists,up 24b76a58-3a69-44b8-b0bb-d061816d6b9b 2026-02-23T08:24:46.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:46.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:46.753 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:46.754 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:46.755 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:46.756 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:24:46.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:24:46.757 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:24:46.757 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:24:46.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5459112f-ba56-4c99-b36d-46f4d249c004 2026-02-23T08:24:46.758 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 5459112f-ba56-4c99-b36d-46f4d249c004' 2026-02-23T08:24:46.758 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 5459112f-ba56-4c99-b36d-46f4d249c004 2026-02-23T08:24:46.758 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:24:46.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDODpxpIfBcLRAA7LY50tlU+IJ8FFf0EZA9cw== 2026-02-23T08:24:46.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDODpxpIfBcLRAA7LY50tlU+IJ8FFf0EZA9cw=="}' 2026-02-23T08:24:46.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5459112f-ba56-4c99-b36d-46f4d249c004 -i td/test-erasure-eio/3/new.json 2026-02-23T08:24:46.995 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:24:47.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:24:47.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDODpxpIfBcLRAA7LY50tlU+IJ8FFf0EZA9cw== --osd-uuid 5459112f-ba56-4c99-b36d-46f4d249c004 2026-02-23T08:24:47.023 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.014+0000 7f91d40e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:47.024 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.014+0000 7f91d40e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:47.025 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.014+0000 7f91d40e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:47.026 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.014+0000 7f91d40e1a40 -1 bdev(0x556b585ee400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:24:47.026 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.014+0000 7f91d40e1a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:24:47.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:24:47.672 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:24:47.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:24:47.674 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:24:47.674 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:24:47.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:24:47.970 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:24:47.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:47.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:24:47.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:24:47.971 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:24:47.993 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.982+0000 7fe13c20ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:47.995 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.986+0000 7fe13c20ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:47.996 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:47.986+0000 7fe13c20ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:48.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:24:48.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:48.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:24:48.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:48.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:48.187 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:48.187 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:24:48.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:48.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:48.188 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:24:48.343 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:48.334+0000 7fe13c20ea40 -1 Falling back to public interface 2026-02-23T08:24:48.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:48.959 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:48.950+0000 7fe13c20ea40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:24:49.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:24:49.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:49.397 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:24:49.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:24:49.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:49.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:24:49.617 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3946620939,v1:127.0.0.1:6827/3946620939] [v2:127.0.0.1:6828/3946620939,v1:127.0.0.1:6829/3946620939] exists,up 5459112f-ba56-4c99-b36d-46f4d249c004 2026-02-23T08:24:49.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:24:49.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:24:49.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:24:49.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:24:49.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:24:49.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:24:49.618 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:49.618 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:49.618 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:49.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:24:49.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:24:49.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:24:49.682 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:24:49.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:24:49.690 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:24:37.482+0000 7f400398ea40 0 load: jerasure load: lrc 2026-02-23T08:24:49.691 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-23T08:24:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-23T08:24:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:24:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:24:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-23T08:24:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:24:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-23T08:24:49.692 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:24:49.693 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-23T08:24:49.926 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:24:49.927 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:24:50.323 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:24:50.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:24:51.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:24:51.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:24:51.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:24:51.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:24:51.337 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:24:51.337 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:24:51.338 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:24:51.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:24:51.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:24:51.338 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:24:51.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:24:51.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:24:51.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:24:51.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:24:51.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:24:51.448 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:24:51.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:24:51.662 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:51.662 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:24:51.662 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:24:51.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:24:51.662 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:51.662 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:24:51.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-23T08:24:51.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-23T08:24:51.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-23T08:24:51.744 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:51.744 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:24:51.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-23T08:24:51.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-23T08:24:51.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-23T08:24:51.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:51.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:24:51.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444035 2026-02-23T08:24:51.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444035 2026-02-23T08:24:51.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444035' 2026-02-23T08:24:51.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:24:51.904 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:24:51.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-23T08:24:51.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-23T08:24:51.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444035 3-107374182403' 2026-02-23T08:24:51.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:51.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-23T08:24:51.984 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:51.985 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:24:51.986 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-23T08:24:51.986 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:51.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-23T08:24:51.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-23T08:24:51.987 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803781 2026-02-23T08:24:51.988 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:52.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-23T08:24:52.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:24:53.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:24:53.208 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:24:53.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-23T08:24:53.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:53.423 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-23T08:24:53.423 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:53.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:24:53.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-23T08:24:53.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:53.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-23T08:24:53.427 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607556 2026-02-23T08:24:53.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-23T08:24:53.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:24:53.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-23T08:24:53.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:53.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444035 2026-02-23T08:24:53.646 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:53.647 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:24:53.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444035 2026-02-23T08:24:53.647 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:53.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444035 2026-02-23T08:24:53.649 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444035 2026-02-23T08:24:53.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444035' 2026-02-23T08:24:53.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:24:53.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444036 -lt 73014444035 2026-02-23T08:24:53.865 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:24:53.866 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182403 2026-02-23T08:24:53.866 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:24:53.867 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:24:53.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-23T08:24:53.868 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:24:53.869 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182403 2026-02-23T08:24:53.870 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182403 2026-02-23T08:24:53.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-23T08:24:53.870 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:24:54.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-23T08:24:54.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:24:54.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:54.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:54.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:24:54.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:24:54.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:24:54.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:24:54.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:24:54.391 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:24:54.391 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:24:54.391 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:24:54.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:24:54.607 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:24:54.607 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:24:54.608 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74343-0 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74343-0 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74343-0 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:24:54.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74343-0 td/test-erasure-eio/ORIGINAL 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74343-0 td/test-erasure-eio 0 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:24:54.922 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-74343-0 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-74343-0 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-0 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-0 2026-02-23T08:24:54.923 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-23T08:24:55.142 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:24:55.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-23T08:24:55.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:24:55.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-23T08:24:55.143 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:24:55.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-23T08:24:55.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:24:55.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:24:55.144 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:24:55.144 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-23T08:24:55.144 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:24:55.145 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:24:55.145 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:55.145 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:55.145 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:55.145 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:24:55.146 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:24:55.205 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:24:55.206 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:24:55.207 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:24:55.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-23T08:24:55.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:24:55.207 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:24:55.207 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:55.207 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:55.208 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:55.208 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:24:55.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:24:55.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-74343-0 0 2026-02-23T08:24:55.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74343-0 2026-02-23T08:24:55.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:24:55.266 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:24:55.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74343-0 2026-02-23T08:24:55.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:24:55.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:24:55.267 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74343-0 td/test-erasure-eio/COPY 2026-02-23T08:24:55.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:24:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:24:55.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-23T08:24:55.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-23T08:24:55.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-23T08:24:55.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74343-0 td/test-erasure-eio 1 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-74343-0 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-23T08:24:55.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:24:55.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-74343-0 2026-02-23T08:24:55.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:24:55.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-0 2026-02-23T08:24:55.294 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-0 2026-02-23T08:24:55.294 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:24:55.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:24:55.514 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:24:55.514 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:24:55.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:24:55.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-23T08:24:55.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:24:55.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-23T08:24:55.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:24:55.515 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-23T08:24:55.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:24:55.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-23T08:24:55.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:24:55.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-23T08:24:55.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:24:55.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:24:55.516 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:24:55.517 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-23T08:24:55.517 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:24:55.517 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:24:55.518 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:55.518 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:55.518 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:55.518 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:24:55.518 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:24:55.580 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:24:55.580 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:24:55.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:24:55.581 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-23T08:24:55.581 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:24:55.581 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:24:55.582 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:55.582 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:55.582 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:55.582 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:24:55.582 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:24:55.582 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-74343-0 1 2026-02-23T08:24:55.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-23T08:24:55.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74343-0 fail 2026-02-23T08:24:55.642 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:24:55.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:24:55.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74343-0 2026-02-23T08:24:55.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-23T08:24:55.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-23T08:24:55.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-74343-0 td/test-erasure-eio/COPY 2026-02-23T08:24:55.661 INFO:tasks.workunit.client.0.trial186.stderr:error getting pool-jerasure/obj-eio-74343-0: (5) Input/output error 2026-02-23T08:24:55.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-23T08:24:55.663 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:24:55.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:24:55.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:24:55.664 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:24:55.927 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:24:55.937 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:24:56.221 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:24:56.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:24:56.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:24:56.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:24:56.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:24:56.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:56.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:56.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:56.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:56.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:56.343 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:56.343 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:24:56.344 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:24:56.345 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:24:56.346 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:24:56.347 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:24:56.347 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:24:56.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:56.348 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:24:56.349 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:24:56.349 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:56.350 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:24:56.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:24:56.351 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:24:56.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:24:56.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:56.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:56.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:24:56.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:24:56.365 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:24:56.365 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:24:56.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:24:56.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:24:56.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:24:56.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:24:56.368 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:24:56.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:24:56.369 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:24:56.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:24:56.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:24:56.370 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:24:56.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:56.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:24:56.372 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:24:56.373 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:24:56.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:24:56.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:24:56.375 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:24:56.376 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:24:56.376 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:56.376 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:56.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:24:56.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:24:56.377 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:24:56.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:24:56.379 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:24:56.379 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:56.379 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:56.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:24:56.381 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:24:56.382 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:24:56.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:24:56.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:56.405 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:56.405 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:56.405 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:56.405 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:56.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:56.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:24:56.433 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:24:56.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:24:56.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:24:56.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:24:56.435 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:24:56.436 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:24:56.436 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:24:56.436 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:24:56.436 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:24:56.437 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:56.437 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:56.437 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:56.437 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:24:56.437 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:24:56.437 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:24:56.493 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:24:56.494 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:24:56.494 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:24:56.494 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:24:56.494 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:24:56.494 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:24:56.494 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:24:56.494 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:24:56.496 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:56.496 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:56.496 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:56.496 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:24:56.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:24:56.496 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:24:56.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:24:56.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:24:56.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:24:56.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:24:56.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:24:56.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:24:56.553 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:24:56.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:24:56.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:56.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:56.667 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:56.667 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:56.667 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:56.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:56.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:24:56.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:24:56.687 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:24:56.688 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:24:56.805 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:24:56.813 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:24:57.815 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:24:57.815 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:24:57.815 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:24:57.815 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:24:57.816 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:57.816 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:57.816 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:24:57.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:24:57.816 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:24:57.867 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:24:57.874 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:24:57.875 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:24:56.414+0000 7f8b29344ec0 0 load: jerasure load: lrc 2026-02-23T08:24:57.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-23T08:24:57.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-23T08:24:57.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-23T08:24:57.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-23T08:24:57.876 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:24:57.876 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-23T08:24:57.877 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:24:57.879 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:24:57.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:24:57.881 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:24:57.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6fd86f18-1823-4ec0-b3ce-29ea1f2cbcac 2026-02-23T08:24:57.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6fd86f18-1823-4ec0-b3ce-29ea1f2cbcac' 2026-02-23T08:24:57.882 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 6fd86f18-1823-4ec0-b3ce-29ea1f2cbcac 2026-02-23T08:24:57.883 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:24:57.895 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDZDpxpLPfsNBAA7nz0rkxM0JGMquvzaut+gw== 2026-02-23T08:24:57.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDZDpxpLPfsNBAA7nz0rkxM0JGMquvzaut+gw=="}' 2026-02-23T08:24:57.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6fd86f18-1823-4ec0-b3ce-29ea1f2cbcac -i td/test-erasure-eio/0/new.json 2026-02-23T08:24:58.007 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:24:58.015 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:24:58.016 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDZDpxpLPfsNBAA7nz0rkxM0JGMquvzaut+gw== --osd-uuid 6fd86f18-1823-4ec0-b3ce-29ea1f2cbcac 2026-02-23T08:24:58.032 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.022+0000 7efc9a0bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:58.034 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.022+0000 7efc9a0bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:58.036 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.026+0000 7efc9a0bfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:58.036 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.026+0000 7efc9a0bfa40 -1 bdev(0x55cbe36de400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:24:58.036 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.026+0000 7efc9a0bfa40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:24:58.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:24:58.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:24:58.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:24:58.722 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:24:58.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:24:58.857 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:24:58.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:24:58.858 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:24:58.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:24:58.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:24:58.870 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:24:58.892 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.882+0000 7fcc143eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:58.893 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.882+0000 7fcc143eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:58.894 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:58.882+0000 7fcc143eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:24:58.987 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:24:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:24:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:24:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:24:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:24:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:24:58.988 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:24:58.988 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:24:58.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:24:58.989 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:24:59.201 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:24:59.275 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:59.266+0000 7fcc143eea40 -1 Falling back to public interface 2026-02-23T08:24:59.861 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:24:59.850+0000 7fcc143eea40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:25:00.202 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:00.203 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:00.203 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:00.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:00.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:00.204 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:00.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:01.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:01.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:01.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:25:01.432 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:01.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:01.432 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:01.650 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1067681161,v1:127.0.0.1:6803/1067681161] [v2:127.0.0.1:6804/1067681161,v1:127.0.0.1:6805/1067681161] exists,up 6fd86f18-1823-4ec0-b3ce-29ea1f2cbcac 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:01.654 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:01.655 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:01.656 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:01.657 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:25:01.658 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:01.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c0818bb6-0841-4ec0-80e6-a87e9ff3256d 2026-02-23T08:25:01.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c0818bb6-0841-4ec0-80e6-a87e9ff3256d' 2026-02-23T08:25:01.659 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 c0818bb6-0841-4ec0-80e6-a87e9ff3256d 2026-02-23T08:25:01.659 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:01.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDdDpxpRZhqJxAAhM1LjiFa2SX57OMoSe8m9Q== 2026-02-23T08:25:01.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDdDpxpRZhqJxAAhM1LjiFa2SX57OMoSe8m9Q=="}' 2026-02-23T08:25:01.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c0818bb6-0841-4ec0-80e6-a87e9ff3256d -i td/test-erasure-eio/1/new.json 2026-02-23T08:25:01.895 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:01.906 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:25:01.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDdDpxpRZhqJxAAhM1LjiFa2SX57OMoSe8m9Q== --osd-uuid c0818bb6-0841-4ec0-80e6-a87e9ff3256d 2026-02-23T08:25:01.931 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:01.922+0000 7f48e6f00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:01.932 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:01.922+0000 7f48e6f00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:01.933 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:01.922+0000 7f48e6f00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:01.934 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:01.926+0000 7f48e6f00a40 -1 bdev(0x5611cf404400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:01.934 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:01.926+0000 7f48e6f00a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:25:02.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:25:02.602 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:02.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:25:02.603 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:25:02.603 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:02.887 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:25:02.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:25:02.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:02.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:02.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:02.888 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:02.909 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:02.898+0000 7f409dabea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:02.911 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:02.902+0000 7f409dabea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:02.912 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:02.902+0000 7f409dabea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:03.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:25:03.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:03.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:25:03.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:03.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:03.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:03.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:03.106 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:03.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:03.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:03.220 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:03.210+0000 7f409dabea40 -1 Falling back to public interface 2026-02-23T08:25:03.325 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:03.817 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:03.806+0000 7f409dabea40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:25:04.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:04.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:04.327 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:04.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:04.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:04.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:04.548 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1361870786,v1:127.0.0.1:6811/1361870786] [v2:127.0.0.1:6812/1361870786,v1:127.0.0.1:6813/1361870786] exists,up c0818bb6-0841-4ec0-80e6-a87e9ff3256d 2026-02-23T08:25:04.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:04.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:04.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:04.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:04.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:04.549 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:04.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:25:04.551 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:04.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=01b75433-602d-488c-b3b3-d36b46a42854 2026-02-23T08:25:04.552 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 01b75433-602d-488c-b3b3-d36b46a42854 2026-02-23T08:25:04.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 01b75433-602d-488c-b3b3-d36b46a42854' 2026-02-23T08:25:04.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:04.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgDpxpZ+E5IRAAlsJe7dMAHiqu4MEznwybYA== 2026-02-23T08:25:04.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgDpxpZ+E5IRAAlsJe7dMAHiqu4MEznwybYA=="}' 2026-02-23T08:25:04.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 01b75433-602d-488c-b3b3-d36b46a42854 -i td/test-erasure-eio/2/new.json 2026-02-23T08:25:04.775 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:04.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:25:04.785 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgDpxpZ+E5IRAAlsJe7dMAHiqu4MEznwybYA== --osd-uuid 01b75433-602d-488c-b3b3-d36b46a42854 2026-02-23T08:25:04.800 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:04.790+0000 7fc687edda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:04.802 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:04.790+0000 7fc687edda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:04.803 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:04.794+0000 7fc687edda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:04.803 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:04.794+0000 7fc687edda40 -1 bdev(0x5615cf108400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:04.804 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:04.794+0000 7fc687edda40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:25:05.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:25:05.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:05.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:25:05.466 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:25:05.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:05.744 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:25:05.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:25:05.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:05.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:05.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:05.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:05.767 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:05.758+0000 7f1f43951a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:05.769 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:05.758+0000 7f1f43951a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:05.770 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:05.762+0000 7f1f43951a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:05.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:25:05.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:05.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:25:05.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:05.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:05.967 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:05.967 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:05.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:05.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:05.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:25:06.178 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:06.204 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:06.194+0000 7f1f43951a40 -1 Falling back to public interface 2026-02-23T08:25:06.784 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:06.774+0000 7f1f43951a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:25:07.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:07.180 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:07.180 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:07.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:07.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:07.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:25:07.398 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1341994648,v1:127.0.0.1:6819/1341994648] [v2:127.0.0.1:6820/1341994648,v1:127.0.0.1:6821/1341994648] exists,up 01b75433-602d-488c-b3b3-d36b46a42854 2026-02-23T08:25:07.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:07.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:07.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:07.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:07.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:07.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:07.403 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:07.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:07.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:07.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:07.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:25:07.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:07.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ddb530df-551e-401f-bba3-ae1193ff58e6 2026-02-23T08:25:07.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 ddb530df-551e-401f-bba3-ae1193ff58e6' 2026-02-23T08:25:07.406 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 ddb530df-551e-401f-bba3-ae1193ff58e6 2026-02-23T08:25:07.407 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:07.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDjDpxpjWdlGBAATM4wqeYKR/ZQLWKB8IPCOg== 2026-02-23T08:25:07.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDjDpxpjWdlGBAATM4wqeYKR/ZQLWKB8IPCOg=="}' 2026-02-23T08:25:07.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ddb530df-551e-401f-bba3-ae1193ff58e6 -i td/test-erasure-eio/3/new.json 2026-02-23T08:25:07.633 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:25:07.643 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:25:07.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDjDpxpjWdlGBAATM4wqeYKR/ZQLWKB8IPCOg== --osd-uuid ddb530df-551e-401f-bba3-ae1193ff58e6 2026-02-23T08:25:07.660 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:07.650+0000 7f529aed0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:07.661 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:07.650+0000 7f529aed0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:07.662 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:07.654+0000 7f529aed0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:07.663 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:07.654+0000 7f529aed0a40 -1 bdev(0x55ff98a64400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:07.663 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:07.654+0000 7f529aed0a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:25:08.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:25:08.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:08.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:25:08.366 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:25:08.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:08.660 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:25:08.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:25:08.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:08.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:08.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:08.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:08.685 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:08.674+0000 7f2b10dbaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:08.686 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:08.678+0000 7f2b10dbaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:08.687 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:08.678+0000 7f2b10dbaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:08.880 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:25:08.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:08.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:25:08.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:08.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:08.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:08.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:08.881 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:08.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:08.882 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:25:09.040 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:09.030+0000 7f2b10dbaa40 -1 Falling back to public interface 2026-02-23T08:25:09.092 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:09.647 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:09.638+0000 7f2b10dbaa40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:25:10.094 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:10.094 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:10.094 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:10.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:10.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:10.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:25:10.308 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 24 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1592706634,v1:127.0.0.1:6827/1592706634] [v2:127.0.0.1:6828/1592706634,v1:127.0.0.1:6829/1592706634] exists,up ddb530df-551e-401f-bba3-ae1193ff58e6 2026-02-23T08:25:10.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:10.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:10.309 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:10.310 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:25:10.310 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:25:10.310 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:25:10.310 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:10.311 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:10.311 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:10.311 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:25:10.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:25:10.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:25:10.366 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:25:10.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:25:10.373 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:24:59.270+0000 7fcc143eea40 0 load: jerasure load: lrc 2026-02-23T08:25:10.373 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:25:10.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-23T08:25:10.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:25:10.661 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:25:11.057 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:25:11.067 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:25:12.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:25:12.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:25:12.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:25:12.068 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:25:12.069 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:25:12.070 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:25:12.071 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:25:12.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:25:12.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:25:12.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:25:12.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:25:12.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:25:12.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:25:12.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:25:12.181 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:25:12.181 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:25:12.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:25:12.403 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:12.404 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:25:12.404 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:25:12.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:25:12.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:12.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:25:12.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-23T08:25:12.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-23T08:25:12.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-23T08:25:12.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:12.482 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:25:12.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-23T08:25:12.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-23T08:25:12.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-23T08:25:12.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:12.565 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:25:12.644 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444035 2026-02-23T08:25:12.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444035 2026-02-23T08:25:12.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444035' 2026-02-23T08:25:12.645 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:12.645 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:25:12.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215108 2026-02-23T08:25:12.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215108 2026-02-23T08:25:12.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444035 3-103079215108' 2026-02-23T08:25:12.725 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:12.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-23T08:25:12.725 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:12.726 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:25:12.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-23T08:25:12.727 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:12.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-23T08:25:12.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-23T08:25:12.729 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803781 2026-02-23T08:25:12.729 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:25:12.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803781 2026-02-23T08:25:12.951 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:25:13.952 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:25:13.953 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:25:14.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-23T08:25:14.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:14.171 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-23T08:25:14.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:14.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:25:14.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-23T08:25:14.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:14.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-23T08:25:14.176 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607556 2026-02-23T08:25:14.176 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-23T08:25:14.176 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:25:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-23T08:25:14.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:14.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444035 2026-02-23T08:25:14.406 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:14.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:25:14.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444035 2026-02-23T08:25:14.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:14.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444035 2026-02-23T08:25:14.410 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444035 2026-02-23T08:25:14.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444035' 2026-02-23T08:25:14.411 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:25:14.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444035 -lt 73014444035 2026-02-23T08:25:14.630 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:14.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-103079215108 2026-02-23T08:25:14.631 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:14.632 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:25:14.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-103079215108 2026-02-23T08:25:14.633 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:14.634 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=103079215108 2026-02-23T08:25:14.635 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 103079215108 2026-02-23T08:25:14.635 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 103079215108' 2026-02-23T08:25:14.635 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:25:14.852 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 103079215108 -lt 103079215108 2026-02-23T08:25:14.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:25:14.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:14.853 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:15.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:25:15.137 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:25:15.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:25:15.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:25:15.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:25:15.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:25:15.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:25:15.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:25:15.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:25:15.356 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:25:15.356 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:15.356 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:15.658 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74343-1 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74343-1 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74343-1 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:25:15.659 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74343-1 td/test-erasure-eio/ORIGINAL 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74343-1 td/test-erasure-eio 1 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-74343-1 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-74343-1 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:25:15.680 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-1 2026-02-23T08:25:15.681 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-1 2026-02-23T08:25:15.681 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:25:15.898 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:25:15.898 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:15.898 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:25:15.898 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:25:15.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-23T08:25:15.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:25:15.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-23T08:25:15.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:25:15.899 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-23T08:25:15.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:25:15.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-23T08:25:15.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:25:15.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-23T08:25:15.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:25:15.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:25:15.901 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:25:15.901 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-23T08:25:15.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:25:15.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:25:15.902 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:15.902 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:15.902 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:15.902 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:25:15.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:25:15.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:25:15.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:25:15.964 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:25:15.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-23T08:25:15.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:25:15.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:25:15.965 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:15.965 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:15.965 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:15.965 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:25:15.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:25:15.966 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-74343-1 1 2026-02-23T08:25:16.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74343-1 2026-02-23T08:25:16.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:25:16.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:25:16.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74343-1 2026-02-23T08:25:16.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:25:16.028 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:25:16.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74343-1 td/test-erasure-eio/COPY 2026-02-23T08:25:16.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:25:16.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:25:16.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-23T08:25:16.051 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-23T08:25:16.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-23T08:25:16.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74343-1 td/test-erasure-eio 2 2026-02-23T08:25:16.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:25:16.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:25:16.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:25:16.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:25:16.053 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-74343-1 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=2 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-74343-1 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:25:16.054 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-1 2026-02-23T08:25:16.055 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-1 2026-02-23T08:25:16.055 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:25:16.267 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=2 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:25:16.268 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/2/type 2026-02-23T08:25:16.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:25:16.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-23T08:25:16.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:25:16.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-23T08:25:16.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:25:16.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:25:16.271 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:25:16.271 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-23T08:25:16.271 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-23T08:25:16.271 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-23T08:25:16.272 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:16.273 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:16.273 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:16.273 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.2.asok 2026-02-23T08:25:16.273 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:25:16.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:25:16.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:25:16.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:25:16.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.2 2026-02-23T08:25:16.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-23T08:25:16.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-23T08:25:16.337 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:16.337 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:16.337 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:16.337 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.2.asok 2026-02-23T08:25:16.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:25:16.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-74343-1 2 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74343-1 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74343-1 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:25:16.398 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74343-1 td/test-erasure-eio/COPY 2026-02-23T08:25:16.419 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:25:16.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:25:16.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:25:16.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:25:16.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:25:16.422 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:25:16.680 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:25:16.689 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:25:16.960 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:25:16.970 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:25:17.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:25:17.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:25:17.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:25:17.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:25:17.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:25:17.085 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:25:17.085 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:25:17.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:17.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:25:17.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:25:17.087 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:17.088 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:25:17.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:25:17.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:25:17.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:25:17.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:17.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:17.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:25:17.104 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:25:17.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:25:17.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:25:17.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:25:17.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:25:17.108 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:25:17.109 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:25:17.109 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:25:17.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:25:17.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:25:17.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:25:17.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:17.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:25:17.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:25:17.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:17.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:25:17.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:25:17.114 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:25:17.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:25:17.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:17.116 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:17.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:25:17.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:25:17.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:25:17.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:25:17.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:25:17.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:17.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:17.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:25:17.121 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:25:17.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:25:17.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:25:17.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:25:17.122 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:25:17.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:25:17.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:25:17.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:25:17.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:25:17.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:25:17.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:25:17.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:25:17.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:17.143 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:17.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:17.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:17.143 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:17.144 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:17.144 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:25:17.172 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:25:17.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:25:17.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:25:17.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:25:17.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:25:17.173 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:25:17.174 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:25:17.174 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:25:17.174 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:25:17.174 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:17.174 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:17.174 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:17.175 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:25:17.175 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:25:17.175 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:25:17.230 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:25:17.231 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:25:17.231 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:25:17.231 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:25:17.231 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:25:17.231 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:25:17.231 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:25:17.231 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:25:17.232 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:17.232 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:17.232 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:17.233 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:25:17.233 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:25:17.233 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:25:17.286 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:25:17.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:25:17.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:25:17.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:25:17.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:25:17.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:25:17.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:25:17.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:25:17.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:17.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:17.399 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:17.400 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:17.400 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:17.400 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:17.400 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:25:17.401 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:25:17.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:25:17.420 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:25:17.541 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:25:17.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:25:18.552 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:25:18.601 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:25:18.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:25:18.609 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:25:17.154+0000 7f7ae1ddfec0 0 load: jerasure load: lrc 2026-02-23T08:25:18.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-23T08:25:18.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-23T08:25:18.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-23T08:25:18.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-23T08:25:18.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:25:18.610 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-23T08:25:18.611 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:18.615 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:18.616 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:18.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:18.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:18.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:18.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:18.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:25:18.618 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:18.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ab5b9896-11b6-443f-b276-e1d923bb459a 2026-02-23T08:25:18.619 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ab5b9896-11b6-443f-b276-e1d923bb459a' 2026-02-23T08:25:18.619 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 ab5b9896-11b6-443f-b276-e1d923bb459a 2026-02-23T08:25:18.619 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:18.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDuDpxpGsoEJRAAPt5R0BHCxS4QdT0ak3P6KA== 2026-02-23T08:25:18.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDuDpxpGsoEJRAAPt5R0BHCxS4QdT0ak3P6KA=="}' 2026-02-23T08:25:18.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ab5b9896-11b6-443f-b276-e1d923bb459a -i td/test-erasure-eio/0/new.json 2026-02-23T08:25:18.737 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:18.745 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:25:18.746 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDuDpxpGsoEJRAAPt5R0BHCxS4QdT0ak3P6KA== --osd-uuid ab5b9896-11b6-443f-b276-e1d923bb459a 2026-02-23T08:25:18.762 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:18.754+0000 7f38cb96ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:18.764 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:18.754+0000 7f38cb96ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:18.765 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:18.754+0000 7f38cb96ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:18.765 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:18.754+0000 7f38cb96ba40 -1 bdev(0x55fece6bc400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:18.765 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:18.754+0000 7f38cb96ba40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:25:19.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:25:19.429 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:19.430 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:25:19.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:25:19.431 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:19.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:25:19.544 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:25:19.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:19.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:19.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:19.544 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:19.559 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:19.550+0000 7f2b7334ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:19.561 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:19.550+0000 7f2b7334ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:19.562 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:19.550+0000 7f2b7334ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:19.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:25:19.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:19.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:25:19.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:19.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:19.697 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:19.697 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:19.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:19.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:19.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:19.812 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:19.996 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:19.986+0000 7f2b7334ea40 -1 Falling back to public interface 2026-02-23T08:25:20.545 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:20.534+0000 7f2b7334ea40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:25:20.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:20.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:20.814 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:20.815 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:20.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:20.815 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:21.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:22.030 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:22.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:22.031 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:22.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:25:22.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:22.032 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:22.239 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4282350415,v1:127.0.0.1:6803/4282350415] [v2:127.0.0.1:6804/4282350415,v1:127.0.0.1:6805/4282350415] exists,up ab5b9896-11b6-443f-b276-e1d923bb459a 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:25:22.240 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:22.241 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:22.242 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:22.242 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:22.243 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:25:22.244 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:22.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=04bd7e85-0962-4937-be6f-4871f66bbc32 2026-02-23T08:25:22.244 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 04bd7e85-0962-4937-be6f-4871f66bbc32' 2026-02-23T08:25:22.244 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 04bd7e85-0962-4937-be6f-4871f66bbc32 2026-02-23T08:25:22.245 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:22.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDyDpxphzHqDhAAZ/E3m6WYxZ7KrsBBZNKYnw== 2026-02-23T08:25:22.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDyDpxphzHqDhAAZ/E3m6WYxZ7KrsBBZNKYnw=="}' 2026-02-23T08:25:22.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 04bd7e85-0962-4937-be6f-4871f66bbc32 -i td/test-erasure-eio/1/new.json 2026-02-23T08:25:22.480 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:22.490 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:25:22.491 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDyDpxphzHqDhAAZ/E3m6WYxZ7KrsBBZNKYnw== --osd-uuid 04bd7e85-0962-4937-be6f-4871f66bbc32 2026-02-23T08:25:22.507 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:22.498+0000 7f943aa40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:22.509 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:22.498+0000 7f943aa40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:22.510 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:22.502+0000 7f943aa40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:22.511 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:22.502+0000 7f943aa40a40 -1 bdev(0x55f0156de400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:22.511 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:22.502+0000 7f943aa40a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:25:23.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:25:23.185 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:23.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:25:23.186 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:25:23.186 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:23.465 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:25:23.465 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:25:23.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:23.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:23.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:23.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:23.489 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:23.478+0000 7f91e191aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:23.490 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:23.482+0000 7f91e191aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:23.492 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:23.482+0000 7f91e191aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:23.685 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:23.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:23.686 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:23.876 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:23.866+0000 7f91e191aa40 -1 Falling back to public interface 2026-02-23T08:25:23.896 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:24.544 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:24.534+0000 7f91e191aa40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:25:24.897 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:24.898 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:24.898 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:24.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:24.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:24.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:25.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:26.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:26.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:26.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:25:26.119 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:26.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:26.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:26.327 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1129082593,v1:127.0.0.1:6811/1129082593] [v2:127.0.0.1:6812/1129082593,v1:127.0.0.1:6813/1129082593] exists,up 04bd7e85-0962-4937-be6f-4871f66bbc32 2026-02-23T08:25:26.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:26.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:26.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:26.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:26.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:26.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:25:26.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:26.334 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a7aaa0eb-65aa-4b05-a94b-a00bf1459f4e 2026-02-23T08:25:26.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 a7aaa0eb-65aa-4b05-a94b-a00bf1459f4e' 2026-02-23T08:25:26.335 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 a7aaa0eb-65aa-4b05-a94b-a00bf1459f4e 2026-02-23T08:25:26.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:26.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD2Dpxp/J4dFBAA2SJ0Qv5nJJTm7SAdp17dfA== 2026-02-23T08:25:26.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD2Dpxp/J4dFBAA2SJ0Qv5nJJTm7SAdp17dfA=="}' 2026-02-23T08:25:26.345 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a7aaa0eb-65aa-4b05-a94b-a00bf1459f4e -i td/test-erasure-eio/2/new.json 2026-02-23T08:25:26.555 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:26.564 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:25:26.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD2Dpxp/J4dFBAA2SJ0Qv5nJJTm7SAdp17dfA== --osd-uuid a7aaa0eb-65aa-4b05-a94b-a00bf1459f4e 2026-02-23T08:25:26.580 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:26.570+0000 7f86d0e81a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:26.581 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:26.570+0000 7f86d0e81a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:26.582 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:26.574+0000 7f86d0e81a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:26.583 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:26.574+0000 7f86d0e81a40 -1 bdev(0x55ff646ca400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:26.583 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:26.574+0000 7f86d0e81a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:25:27.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:25:27.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:27.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:25:27.174 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:25:27.175 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:27.480 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:25:27.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:25:27.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:27.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:27.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:27.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:27.496 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:27.486+0000 7f63f5e74a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:27.497 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:27.486+0000 7f63f5e74a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:27.498 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:27.490+0000 7f63f5e74a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:27.698 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:25:27.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:27.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:25:27.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:27.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:27.699 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:27.699 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:27.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:27.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:27.700 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:25:27.908 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:27.916 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:27.905+0000 7f63f5e74a40 -1 Falling back to public interface 2026-02-23T08:25:28.523 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:28.513+0000 7f63f5e74a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:25:28.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:28.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:28.910 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:28.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:28.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:28.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3532914683,v1:127.0.0.1:6819/3532914683] [v2:127.0.0.1:6820/3532914683,v1:127.0.0.1:6821/3532914683] exists,up a7aaa0eb-65aa-4b05-a94b-a00bf1459f4e 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:29.118 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:29.119 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:29.120 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:25:29.122 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:29.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6dee0f04-ea6c-4b47-88bf-35ed4f1f09f2 2026-02-23T08:25:29.123 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 6dee0f04-ea6c-4b47-88bf-35ed4f1f09f2' 2026-02-23T08:25:29.123 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 6dee0f04-ea6c-4b47-88bf-35ed4f1f09f2 2026-02-23T08:25:29.123 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:29.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD5DpxpRCqwBxAAqZm0tv8CsV3HWcnhGkQUzg== 2026-02-23T08:25:29.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD5DpxpRCqwBxAAqZm0tv8CsV3HWcnhGkQUzg=="}' 2026-02-23T08:25:29.136 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6dee0f04-ea6c-4b47-88bf-35ed4f1f09f2 -i td/test-erasure-eio/3/new.json 2026-02-23T08:25:29.357 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:25:29.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:25:29.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5DpxpRCqwBxAAqZm0tv8CsV3HWcnhGkQUzg== --osd-uuid 6dee0f04-ea6c-4b47-88bf-35ed4f1f09f2 2026-02-23T08:25:29.384 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:29.373+0000 7fd283cbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:29.385 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:29.373+0000 7fd283cbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:29.386 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:29.373+0000 7fd283cbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:29.386 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:29.377+0000 7fd283cbfa40 -1 bdev(0x563bf560a400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:29.386 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:29.377+0000 7fd283cbfa40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:25:30.012 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:25:30.013 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:30.014 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:25:30.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:25:30.014 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:30.303 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:25:30.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:25:30.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:30.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:30.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:30.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:30.326 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:30.317+0000 7f922605fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:30.328 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:30.317+0000 7f922605fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:30.329 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:30.317+0000 7f922605fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:30.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:25:30.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:30.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:25:30.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:30.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:30.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:30.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:30.519 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:30.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:30.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:25:30.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:30.784 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:30.773+0000 7f922605fa40 -1 Falling back to public interface 2026-02-23T08:25:31.444 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:31.433+0000 7f922605fa40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:25:31.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:31.723 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:31.723 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:31.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:31.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:31.724 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:25:31.946 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:32.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:32.948 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:32.948 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:32.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:25:32.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:32.949 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:25:33.168 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1609532058,v1:127.0.0.1:6827/1609532058] [v2:127.0.0.1:6828/1609532058,v1:127.0.0.1:6829/1609532058] exists,up 6dee0f04-ea6c-4b47-88bf-35ed4f1f09f2 2026-02-23T08:25:33.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:33.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:33.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:33.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:25:33.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:25:33.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:25:33.170 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:33.170 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:33.170 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:33.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:25:33.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:25:33.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:25:33.226 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:25:33.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:25:33.233 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:25:19.990+0000 7f2b7334ea40 0 load: jerasure load: lrc 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:25:33.234 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-23T08:25:33.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:25:33.513 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:25:33.892 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:25:33.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:25:34.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:25:34.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:25:34.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:25:34.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:25:34.905 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:25:34.905 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:25:34.905 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:25:34.906 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:25:34.906 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:25:34.907 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:25:35.002 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:25:35.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:25:35.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:25:35.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:25:35.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:25:35.003 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:25:35.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:25:35.232 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:35.232 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:25:35.232 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:25:35.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:25:35.232 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:35.232 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:25:35.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-23T08:25:35.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-23T08:25:35.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-23T08:25:35.311 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:35.311 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:25:35.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-23T08:25:35.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-23T08:25:35.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-23T08:25:35.392 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:35.393 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:25:35.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-23T08:25:35.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-23T08:25:35.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628' 2026-02-23T08:25:35.472 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:35.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:25:35.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-23T08:25:35.554 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-23T08:25:35.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628 3-115964116995' 2026-02-23T08:25:35.555 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:35.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-23T08:25:35.555 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:35.556 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:25:35.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-23T08:25:35.557 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:35.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-23T08:25:35.559 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803781 2026-02-23T08:25:35.559 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-23T08:25:35.559 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:25:35.770 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-23T08:25:35.770 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:25:36.772 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:25:36.772 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:25:36.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-23T08:25:36.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:36.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-23T08:25:36.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:36.996 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:25:36.997 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-23T08:25:36.997 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:36.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-23T08:25:36.999 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574853 2026-02-23T08:25:36.999 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-23T08:25:36.999 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:25:37.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574853 2026-02-23T08:25:37.212 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:37.213 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378628 2026-02-23T08:25:37.213 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:37.214 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:25:37.215 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378628 2026-02-23T08:25:37.215 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:37.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378628 2026-02-23T08:25:37.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-23T08:25:37.217 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 81604378628 2026-02-23T08:25:37.217 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:25:37.445 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-23T08:25:37.445 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:37.445 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116995 2026-02-23T08:25:37.446 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:37.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:25:37.448 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-23T08:25:37.448 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:37.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116995 2026-02-23T08:25:37.449 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 115964116995 2026-02-23T08:25:37.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-23T08:25:37.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:25:37.680 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-23T08:25:37.680 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:25:37.681 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:37.681 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:37.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:25:37.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:25:37.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:25:37.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:25:37.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:25:37.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:25:37.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:25:37.974 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:25:38.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:25:38.192 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:25:38.192 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:38.192 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:38.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:25:38.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:25:38.480 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-23T08:25:38.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-74343-1 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-74343-1 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-74343-1 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:25:38.482 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:25:38.483 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-74343-1 td/test-erasure-eio/ORIGINAL 2026-02-23T08:25:38.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-74343-1 td/test-erasure-eio 1 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-74343-1 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-23T08:25:38.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-23T08:25:38.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-23T08:25:38.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-23T08:25:38.501 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-74343-1 2026-02-23T08:25:38.501 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:25:38.501 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-74343-1 2026-02-23T08:25:38.501 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-74343-1 2026-02-23T08:25:38.501 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:25:38.719 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-74343-1 remove 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-74343-1 remove 2026-02-23T08:25:38.720 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:25:38.721 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-74343-1 remove 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-23T08:25:39.029 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-74343-1 remove 2026-02-23T08:25:39.209 INFO:tasks.workunit.client.0.trial186.stdout:remove 1#2:13243a9f:::obj-remove-74343-1:head# 2026-02-23T08:25:39.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-23T08:25:39.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:39.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:25:39.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-23T08:25:39.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:25:39.293 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:25:39.294 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:39.295 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:39.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:25:39.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:25:39.296 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:25:39.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-23T08:25:39.297 INFO:tasks.workunit.client.0.trial186.stderr:start osd.1 2026-02-23T08:25:39.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:39.297 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-23T08:25:39.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-23T08:25:39.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:25:39.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:25:39.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:25:39.313 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:39.301+0000 7f646060ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:39.314 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:39.305+0000 7f646060ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:39.315 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:39.305+0000 7f646060ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:39.508 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:39.672 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:39.661+0000 7f646060ea40 -1 Falling back to public interface 2026-02-23T08:25:39.727 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:40.321 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:40.309+0000 7f646060ea40 -1 osd.1 34 log_to_monitors true 2026-02-23T08:25:40.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:40.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:40.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:40.728 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:40.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:40.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:40.946 INFO:tasks.workunit.client.0.trial186.stderr:osd.1 up in weight 1 up_from 38 up_thru 38 down_at 35 last_clean_interval [13,34) [v2:127.0.0.1:6810/2674902562,v1:127.0.0.1:6811/2674902562] [v2:127.0.0.1:6812/2674902562,v1:127.0.0.1:6813/2674902562] exists,up 04bd7e85-0962-4937-be6f-4871f66bbc32 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:25:40.947 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:25:40.948 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:25:40.948 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:25:40.948 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:25:40.948 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:25:40.948 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:25:41.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:25:41.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:25:41.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:25:41.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:25:41.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:25:41.050 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:25:41.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:25:41.278 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:41.279 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:25:41.279 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:25:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:25:41.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:41.279 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:25:41.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-23T08:25:41.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-23T08:25:41.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-23T08:25:41.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:41.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:25:41.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757251 2026-02-23T08:25:41.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757251 2026-02-23T08:25:41.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-163208757251' 2026-02-23T08:25:41.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:41.438 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:25:41.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378631 2026-02-23T08:25:41.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378631 2026-02-23T08:25:41.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-163208757251 2-81604378631' 2026-02-23T08:25:41.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:41.518 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:25:41.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116998 2026-02-23T08:25:41.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116998 2026-02-23T08:25:41.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-163208757251 2-81604378631 3-115964116998' 2026-02-23T08:25:41.596 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:41.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-23T08:25:41.597 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:41.598 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:25:41.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-23T08:25:41.599 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:41.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-23T08:25:41.600 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-23T08:25:41.601 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803784 2026-02-23T08:25:41.601 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:25:41.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803784 2026-02-23T08:25:41.819 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:41.820 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-163208757251 2026-02-23T08:25:41.820 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:41.821 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:25:41.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-163208757251 2026-02-23T08:25:41.822 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:41.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757251 2026-02-23T08:25:41.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 163208757251' 2026-02-23T08:25:41.824 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 163208757251 2026-02-23T08:25:41.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:25:42.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-23T08:25:42.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:42.044 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378631 2026-02-23T08:25:42.044 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:42.045 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:25:42.046 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378631 2026-02-23T08:25:42.046 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:42.047 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378631 2026-02-23T08:25:42.048 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378631' 2026-02-23T08:25:42.048 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 81604378631 2026-02-23T08:25:42.048 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:25:42.268 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378631 -lt 81604378631 2026-02-23T08:25:42.269 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:42.269 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116998 2026-02-23T08:25:42.269 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:42.270 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:25:42.271 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116998 2026-02-23T08:25:42.271 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:42.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116998 2026-02-23T08:25:42.273 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-23T08:25:42.273 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964116998 2026-02-23T08:25:42.273 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:25:42.501 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116998 -lt 115964116998 2026-02-23T08:25:42.501 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:25:42.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:42.502 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:42.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:25:42.786 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:25:42.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:25:42.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:25:42.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:25:42.786 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:25:42.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:25:42.787 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:25:42.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:25:42.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:25:42.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:42.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:43.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-74343-1 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-74343-1 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:25:43.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-74343-1 td/test-erasure-eio/COPY 2026-02-23T08:25:43.297 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:25:43.298 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:25:43.299 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-23T08:25:43.299 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-74343-1 td/test-erasure-eio 2 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-23T08:25:43.302 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-74343-1 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-74343-1 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:25:43.303 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-74343-1 2026-02-23T08:25:43.304 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-74343-1 2026-02-23T08:25:43.304 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:25:43.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:25:43.514 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:43.514 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:25:43.514 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:25:43.514 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-74343-1 remove 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-74343-1 remove 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-23T08:25:43.515 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:25:43.516 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:25:43.516 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:25:43.516 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:25:43.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:25:43.517 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:25:43.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:25:43.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-74343-1 remove 2026-02-23T08:25:43.823 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-23T08:25:43.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-23T08:25:43.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-23T08:25:43.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-23T08:25:43.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-23T08:25:43.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-74343-1 remove 2026-02-23T08:25:44.047 INFO:tasks.workunit.client.0.trial186.stdout:remove 2#2:13243a9f:::obj-remove-74343-1:head# 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:44.151 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:44.152 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:25:44.153 INFO:tasks.workunit.client.0.trial186.stderr:start osd.2 2026-02-23T08:25:44.154 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:44.154 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:25:44.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:25:44.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:25:44.155 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:25:44.156 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:25:44.173 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:44.161+0000 7f46d110ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:44.174 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:44.165+0000 7f46d110ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:44.176 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:44.165+0000 7f46d110ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:44.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:0 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:44.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:25:44.520 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:44.509+0000 7f46d110ba40 -1 Falling back to public interface 2026-02-23T08:25:44.582 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:45.160 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:45.149+0000 7f46d110ba40 -1 osd.2 39 log_to_monitors true 2026-02-23T08:25:45.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:45.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:45.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:45.583 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:45.584 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:45.584 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:25:45.805 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 up in weight 1 up_from 43 up_thru 35 down_at 40 last_clean_interval [19,39) [v2:127.0.0.1:6818/1559927357,v1:127.0.0.1:6819/1559927357] [v2:127.0.0.1:6820/1559927357,v1:127.0.0.1:6821/1559927357] exists,up a7aaa0eb-65aa-4b05-a94b-a00bf1459f4e 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:25:45.806 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:25:45.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:25:45.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:25:45.807 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:25:45.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:25:45.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:25:45.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:25:45.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:25:45.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:25:45.907 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:25:46.116 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:25:46.116 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:25:46.117 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:25:46.117 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:25:46.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:25:46.117 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:46.117 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:25:46.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-23T08:25:46.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-23T08:25:46.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-23T08:25:46.198 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:46.198 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:25:46.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757254 2026-02-23T08:25:46.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757254 2026-02-23T08:25:46.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-163208757254' 2026-02-23T08:25:46.275 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:46.275 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:25:46.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-23T08:25:46.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-23T08:25:46.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-163208757254 2-184683593731' 2026-02-23T08:25:46.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:25:46.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:25:46.436 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117001 2026-02-23T08:25:46.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117001 2026-02-23T08:25:46.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-163208757254 2-184683593731 3-115964117001' 2026-02-23T08:25:46.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:46.438 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-23T08:25:46.438 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:46.438 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:25:46.439 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-23T08:25:46.439 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:46.441 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-23T08:25:46.441 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-23T08:25:46.441 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.0 seq 25769803787 2026-02-23T08:25:46.441 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:25:46.668 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-23T08:25:46.669 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:25:47.670 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:25:47.670 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:25:47.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803787 2026-02-23T08:25:47.881 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:47.882 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-163208757254 2026-02-23T08:25:47.882 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:47.883 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:25:47.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-163208757254 2026-02-23T08:25:47.884 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:47.885 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=163208757254 2026-02-23T08:25:47.886 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 163208757254' 2026-02-23T08:25:47.886 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.1 seq 163208757254 2026-02-23T08:25:47.886 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:25:48.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 163208757254 -lt 163208757254 2026-02-23T08:25:48.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:48.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-184683593731 2026-02-23T08:25:48.111 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:48.112 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:25:48.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-23T08:25:48.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:48.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593731 2026-02-23T08:25:48.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-23T08:25:48.115 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.2 seq 184683593731 2026-02-23T08:25:48.115 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:25:48.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-23T08:25:48.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:25:48.332 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117001 2026-02-23T08:25:48.333 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:25:48.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:25:48.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117001 2026-02-23T08:25:48.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:25:48.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117001 2026-02-23T08:25:48.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117001' 2026-02-23T08:25:48.336 INFO:tasks.workunit.client.0.trial186.stderr:waiting osd.3 seq 115964117001 2026-02-23T08:25:48.336 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:25:48.565 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117002 -lt 115964117001 2026-02-23T08:25:48.566 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:25:48.566 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:48.566 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:48.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:25:48.859 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:25:48.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:25:48.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:25:48.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:25:48.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:25:48.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:25:48.860 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:25:49.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:25:49.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:25:49.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:25:49.077 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-74343-1 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-74343-1 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:25:49.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-74343-1 td/test-erasure-eio/COPY 2026-02-23T08:25:49.387 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:25:49.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:25:49.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-23T08:25:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:25:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:25:49.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:25:49.674 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:25:49.684 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:25:49.957 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:25:49.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:25:49.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:25:49.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:25:49.968 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:25:49.968 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:25:49.968 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:25:49.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:25:49.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:25:49.969 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:25:50.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:25:50.080 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:25:50.081 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:25:50.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:25:50.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:25:50.083 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:25:50.083 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:25:50.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:50.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:25:50.084 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:25:50.085 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:50.086 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:25:50.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:25:50.087 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:25:50.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:25:50.097 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:50.098 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:50.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:25:50.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:25:50.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:25:50.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:25:50.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:25:50.099 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:25:50.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:25:50.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:25:50.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:25:50.100 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:25:50.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:25:50.100 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:25:50.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:25:50.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:25:50.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:25:50.102 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:25:50.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:25:50.103 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:25:50.103 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:25:50.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:25:50.104 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:25:50.105 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:25:50.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:50.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:25:50.106 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:25:50.106 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:25:50.107 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:25:50.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:25:50.108 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:25:50.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:25:50.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:50.110 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:50.110 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:25:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:25:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:25:50.111 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:25:50.112 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:25:50.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:50.113 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:50.113 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:25:50.114 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:25:50.115 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:25:50.137 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:25:50.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:50.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:50.138 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:50.138 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:50.138 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:50.138 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:50.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:25:50.163 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:25:50.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:25:50.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:25:50.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:25:50.163 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:25:50.164 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:25:50.165 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:25:50.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:25:50.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:25:50.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:25:50.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:25:50.223 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:25:50.223 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:25:50.223 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:25:50.223 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:25:50.224 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:50.224 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:50.224 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:50.224 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:25:50.224 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:25:50.224 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:25:50.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:25:50.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:25:50.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:25:50.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:25:50.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:25:50.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:25:50.281 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:25:50.397 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:25:50.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:50.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:50.398 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:50.398 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:50.398 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:50.398 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:50.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:25:50.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:25:50.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:25:50.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:25:50.530 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:25:50.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:25:51.540 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:25:51.541 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:25:51.593 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:25:51.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:25:51.600 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:25:50.145+0000 7f769d5f9ec0 0 load: jerasure load: lrc 2026-02-23T08:25:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-23T08:25:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-23T08:25:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-23T08:25:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-23T08:25:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-23T08:25:51.601 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:25:51.601 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-23T08:25:51.602 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:51.604 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:51.605 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:25:51.606 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:51.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a657ceb4-6b73-40fa-ac36-f61208c1cc27 2026-02-23T08:25:51.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a657ceb4-6b73-40fa-ac36-f61208c1cc27' 2026-02-23T08:25:51.607 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 a657ceb4-6b73-40fa-ac36-f61208c1cc27 2026-02-23T08:25:51.607 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:51.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAPD5xpYM5nJBAAzRUBnb8jtpvIMbCAb548Nw== 2026-02-23T08:25:51.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAPD5xpYM5nJBAAzRUBnb8jtpvIMbCAb548Nw=="}' 2026-02-23T08:25:51.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a657ceb4-6b73-40fa-ac36-f61208c1cc27 -i td/test-erasure-eio/0/new.json 2026-02-23T08:25:51.733 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:51.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:25:51.742 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAPD5xpYM5nJBAAzRUBnb8jtpvIMbCAb548Nw== --osd-uuid a657ceb4-6b73-40fa-ac36-f61208c1cc27 2026-02-23T08:25:51.758 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:51.745+0000 7f7d0e2cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:51.759 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:51.749+0000 7f7d0e2cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:51.760 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:51.749+0000 7f7d0e2cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:51.761 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:51.749+0000 7f7d0e2cca40 -1 bdev(0x55db1cadc400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:51.761 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:51.749+0000 7f7d0e2cca40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:25:52.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:25:52.466 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:52.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:25:52.468 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:25:52.468 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:52.586 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:25:52.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:25:52.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:52.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:52.599 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:52.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:52.651 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:52.637+0000 7f4653823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:52.652 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:52.641+0000 7f4653823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:52.653 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:52.641+0000 7f4653823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:52.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:25:52.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:52.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:25:52.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:52.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:52.747 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:52.748 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:52.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:52.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:52.748 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:52.860 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:53.025 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:53.013+0000 7f4653823a40 -1 Falling back to public interface 2026-02-23T08:25:53.572 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:53.561+0000 7f4653823a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:25:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:53.862 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:53.863 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:53.863 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:54.074 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:55.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:55.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:55.076 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:25:55.076 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:55.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:55.077 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:25:55.287 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2407618450,v1:127.0.0.1:6803/2407618450] [v2:127.0.0.1:6804/2407618450,v1:127.0.0.1:6805/2407618450] exists,up a657ceb4-6b73-40fa-ac36-f61208c1cc27 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:55.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:55.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:55.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:25:55.291 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:55.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7c54efc6-1d61-4f8a-9b85-c8269745ee54 2026-02-23T08:25:55.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7c54efc6-1d61-4f8a-9b85-c8269745ee54' 2026-02-23T08:25:55.292 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 7c54efc6-1d61-4f8a-9b85-c8269745ee54 2026-02-23T08:25:55.292 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:55.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQATD5xp2xK/ERAAu46N1NSc0pCcXaBtj49DhQ== 2026-02-23T08:25:55.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQATD5xp2xK/ERAAu46N1NSc0pCcXaBtj49DhQ=="}' 2026-02-23T08:25:55.305 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7c54efc6-1d61-4f8a-9b85-c8269745ee54 -i td/test-erasure-eio/1/new.json 2026-02-23T08:25:55.529 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:55.538 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:25:55.539 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQATD5xp2xK/ERAAu46N1NSc0pCcXaBtj49DhQ== --osd-uuid 7c54efc6-1d61-4f8a-9b85-c8269745ee54 2026-02-23T08:25:55.556 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:55.545+0000 7f50b0b95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:55.557 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:55.545+0000 7f50b0b95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:55.558 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:55.545+0000 7f50b0b95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:55.559 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:55.549+0000 7f50b0b95a40 -1 bdev(0x5575b76e2400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:55.559 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:55.549+0000 7f50b0b95a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:25:56.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:25:56.182 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:56.183 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:25:56.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:25:56.183 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:56.483 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:25:56.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:25:56.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:56.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:56.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:56.485 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:56.506 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:56.493+0000 7fcd4d07ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:56.508 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:56.497+0000 7fcd4d07ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:56.509 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:56.497+0000 7fcd4d07ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:56.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:25:56.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:56.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:25:56.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:56.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:56.702 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:56.703 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:56.703 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:56.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:56.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:56.872 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:56.861+0000 7fcd4d07ea40 -1 Falling back to public interface 2026-02-23T08:25:56.911 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:25:57.466 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:57.453+0000 7fcd4d07ea40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:25:57.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:25:57.913 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:57.913 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:25:57.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:25:57.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:57.914 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:25:58.129 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3663906921,v1:127.0.0.1:6811/3663906921] [v2:127.0.0.1:6812/3663906921,v1:127.0.0.1:6813/3663906921] exists,up 7c54efc6-1d61-4f8a-9b85-c8269745ee54 2026-02-23T08:25:58.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:25:58.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:25:58.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:25:58.130 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:25:58.131 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:25:58.132 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:25:58.132 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:25:58.132 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:25:58.132 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:58.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:25:58.132 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:25:58.133 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:25:58.134 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5a91b172-e692-4d4c-8a50-88f049a2dad1 2026-02-23T08:25:58.135 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 5a91b172-e692-4d4c-8a50-88f049a2dad1 2026-02-23T08:25:58.135 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5a91b172-e692-4d4c-8a50-88f049a2dad1' 2026-02-23T08:25:58.135 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:25:58.147 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAWD5xpN7dLCBAAd+Kcsj0kr1tWfj7b+Be+Ng== 2026-02-23T08:25:58.147 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAWD5xpN7dLCBAAd+Kcsj0kr1tWfj7b+Be+Ng=="}' 2026-02-23T08:25:58.147 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5a91b172-e692-4d4c-8a50-88f049a2dad1 -i td/test-erasure-eio/2/new.json 2026-02-23T08:25:58.375 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:25:58.385 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:25:58.386 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAWD5xpN7dLCBAAd+Kcsj0kr1tWfj7b+Be+Ng== --osd-uuid 5a91b172-e692-4d4c-8a50-88f049a2dad1 2026-02-23T08:25:58.403 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:58.393+0000 7f5d919e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:58.405 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:58.393+0000 7f5d919e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:58.406 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:58.393+0000 7f5d919e9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:58.407 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:58.397+0000 7f5d919e9a40 -1 bdev(0x56257ce6e400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:25:58.407 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:58.397+0000 7f5d919e9a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:25:59.050 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:25:59.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:25:59.051 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:25:59.052 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:25:59.052 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:25:59.352 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:25:59.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:25:59.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:25:59.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:25:59.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:25:59.353 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:25:59.375 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:59.365+0000 7faea778ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:59.377 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:59.365+0000 7faea778ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:59.378 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:59.365+0000 7faea778ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:25:59.561 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:25:59.732 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:25:59.721+0000 7faea778ca40 -1 Falling back to public interface 2026-02-23T08:25:59.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:00.428 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:00.417+0000 7faea778ca40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:26:00.781 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:00.781 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:00.782 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:00.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:00.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:00.783 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:01.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:02.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:02.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:02.006 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:26:02.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:26:02.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:02.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:02.218 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2181012917,v1:127.0.0.1:6819/2181012917] [v2:127.0.0.1:6820/2181012917,v1:127.0.0.1:6821/2181012917] exists,up 5a91b172-e692-4d4c-8a50-88f049a2dad1 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:26:02.220 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:26:02.221 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:02.221 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:26:02.222 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:26:02.223 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:26:02.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8c429929-35f8-4d3c-bae6-4b2a8cbf5a52 2026-02-23T08:26:02.224 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 8c429929-35f8-4d3c-bae6-4b2a8cbf5a52 2026-02-23T08:26:02.224 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 8c429929-35f8-4d3c-bae6-4b2a8cbf5a52' 2026-02-23T08:26:02.224 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:26:02.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAaD5xp1L+rDRAAorOvKvFCXLijZKGjxt+tBg== 2026-02-23T08:26:02.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAaD5xp1L+rDRAAorOvKvFCXLijZKGjxt+tBg=="}' 2026-02-23T08:26:02.237 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8c429929-35f8-4d3c-bae6-4b2a8cbf5a52 -i td/test-erasure-eio/3/new.json 2026-02-23T08:26:02.466 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:26:02.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:26:02.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAaD5xp1L+rDRAAorOvKvFCXLijZKGjxt+tBg== --osd-uuid 8c429929-35f8-4d3c-bae6-4b2a8cbf5a52 2026-02-23T08:26:02.501 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:02.489+0000 7f818f0bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:02.503 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:02.493+0000 7f818f0bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:02.504 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:02.493+0000 7f818f0bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:02.504 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:02.493+0000 7f818f0bca40 -1 bdev(0x55b29584e400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:26:02.505 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:02.493+0000 7f818f0bca40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:26:03.158 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:26:03.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:26:03.159 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:26:03.160 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:26:03.160 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:26:03.458 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:26:03.459 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:26:03.459 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:26:03.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:26:03.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:26:03.460 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:26:03.482 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:03.469+0000 7f8ffa26ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:03.484 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:03.473+0000 7f8ffa26ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:03.485 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:03.473+0000 7f8ffa26ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:26:03.676 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:03.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:03.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:26:03.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:03.956 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:03.945+0000 7f8ffa26ba40 -1 Falling back to public interface 2026-02-23T08:26:04.568 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:04.557+0000 7f8ffa26ba40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:26:04.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:04.901 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:04.901 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:04.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:04.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:04.902 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:26:05.139 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:06.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:06.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:06.140 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:26:06.141 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:26:06.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:06.141 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:26:06.357 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3568937568,v1:127.0.0.1:6827/3568937568] [v2:127.0.0.1:6828/3568937568,v1:127.0.0.1:6829/3568937568] exists,up 8c429929-35f8-4d3c-bae6-4b2a8cbf5a52 2026-02-23T08:26:06.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:06.359 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:26:06.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:26:06.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:26:06.415 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:26:06.421 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:26:06.423 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:25:53.017+0000 7f4653823a40 0 load: jerasure load: lrc 2026-02-23T08:26:06.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-23T08:26:06.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-23T08:26:06.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:26:06.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:26:06.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-23T08:26:06.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:26:06.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-23T08:26:06.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:26:06.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-23T08:26:06.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:26:06.707 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:26:07.069 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:26:07.078 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:26:08.079 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:26:08.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:26:08.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:26:08.080 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:26:08.080 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:26:08.081 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:26:08.081 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:26:08.081 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:26:08.081 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:26:08.082 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:26:08.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:26:08.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:26:08.190 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:26:08.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:26:08.191 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:26:08.191 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:26:08.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:26:08.403 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:08.403 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:26:08.403 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:26:08.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:26:08.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:08.403 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:26:08.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-23T08:26:08.486 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-23T08:26:08.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-23T08:26:08.487 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:08.487 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:26:08.567 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-23T08:26:08.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-23T08:26:08.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607556' 2026-02-23T08:26:08.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:08.568 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:26:08.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444036 2026-02-23T08:26:08.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444036 2026-02-23T08:26:08.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607556 2-73014444036' 2026-02-23T08:26:08.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:08.650 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:26:08.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-23T08:26:08.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-23T08:26:08.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607556 2-73014444036 3-107374182403' 2026-02-23T08:26:08.728 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:08.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-23T08:26:08.728 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:08.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:26:08.731 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-23T08:26:08.731 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:08.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-23T08:26:08.732 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-23T08:26:08.732 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803782 2026-02-23T08:26:08.732 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:26:08.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-23T08:26:08.942 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:08.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-23T08:26:08.943 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:08.944 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:26:08.945 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-23T08:26:08.945 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:08.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-23T08:26:08.947 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607556 2026-02-23T08:26:08.947 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-23T08:26:08.947 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:26:09.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607557 -lt 51539607556 2026-02-23T08:26:09.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:09.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444036 2026-02-23T08:26:09.170 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:09.171 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:26:09.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444036 2026-02-23T08:26:09.172 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:09.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444036 2026-02-23T08:26:09.174 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 73014444036 2026-02-23T08:26:09.174 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-23T08:26:09.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:26:09.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-23T08:26:09.388 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:09.388 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182403 2026-02-23T08:26:09.389 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:09.390 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:26:09.391 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-23T08:26:09.391 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:09.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182403 2026-02-23T08:26:09.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-23T08:26:09.393 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182403 2026-02-23T08:26:09.393 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:26:09.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182401 -lt 107374182403 2026-02-23T08:26:09.610 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:26:10.611 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:26:10.612 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:26:10.822 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-23T08:26:10.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:26:10.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:10.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:11.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:26:11.101 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:26:11.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:26:11.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:26:11.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:26:11.101 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:26:11.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:26:11.102 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:26:11.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:26:11.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:26:11.307 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:11.308 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:11.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:26:11.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:26:11.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:26:11.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-23T08:26:11.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-23T08:26:11.606 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74343-0 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74343-0 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:26:11.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74343-0 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:26:11.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74343-0 td/test-erasure-eio/ORIGINAL 2026-02-23T08:26:11.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74343-0 td/test-erasure-eio 0 2026-02-23T08:26:11.626 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-74343-0 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:26:11.627 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:26:11.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-23T08:26:11.628 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:26:11.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-74343-0 2026-02-23T08:26:11.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:26:11.628 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-0 2026-02-23T08:26:11.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-0 2026-02-23T08:26:11.628 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:26:11.842 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:26:11.843 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-23T08:26:11.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:26:11.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-23T08:26:11.844 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:26:11.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-23T08:26:11.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:26:11.845 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:26:11.845 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:26:11.846 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-23T08:26:11.846 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:26:11.846 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:26:11.846 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:11.846 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:11.846 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:11.847 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:26:11.847 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:26:11.909 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:26:11.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:26:11.910 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:26:11.910 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-23T08:26:11.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-23T08:26:11.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-23T08:26:11.911 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:11.911 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:11.911 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:11.911 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.3.asok 2026-02-23T08:26:11.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:26:11.912 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-74343-0 0 2026-02-23T08:26:11.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74343-0 2026-02-23T08:26:11.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:26:11.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:26:11.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74343-0 2026-02-23T08:26:11.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:26:11.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:26:11.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74343-0 td/test-erasure-eio/COPY 2026-02-23T08:26:11.993 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:26:11.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:26:11.995 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-23T08:26:11.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-74343-0 2026-02-23T08:26:11.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:26:11.996 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-0 2026-02-23T08:26:11.996 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-0 2026-02-23T08:26:11.996 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:26:12.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:26:12.218 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:12.218 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:26:12.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:26:12.219 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:26:12.527 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:26:12.527 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-23T08:26:12.777 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already out. 2026-02-23T08:26:12.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-74343-0 2026-02-23T08:26:12.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:26:12.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-0 2026-02-23T08:26:12.788 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-23T08:26:12.789 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-0 2026-02-23T08:26:12.789 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:26:13.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:26:13.007 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:13.007 INFO:tasks.workunit.client.0.trial186.stderr:0' 2026-02-23T08:26:13.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-23T08:26:13.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-23T08:26:13.268 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already in. 2026-02-23T08:26:13.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-23T08:26:13.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:26:13.278 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:13.279 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:26:13.280 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:26:13.281 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:26:13.281 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:26:13.281 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:26:13.282 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:26:13.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:26:13.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:26:13.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:26:13.283 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:26:13.304 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:13.293+0000 7f81151c3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:13.306 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:13.293+0000 7f81151c3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:13.307 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:13.293+0000 7f81151c3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:13.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:26:13.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:26:13.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:26:13.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:26:13.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:26:13.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:13.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:26:13.500 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:13.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:13.500 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:13.689 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:13.677+0000 7f81151c3a40 -1 Falling back to public interface 2026-02-23T08:26:13.715 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:14.326 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:14.313+0000 7f81151c3a40 -1 osd.2 32 log_to_monitors true 2026-02-23T08:26:14.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:14.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:14.716 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:14.717 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:14.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:14.717 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:14.929 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 41 up_thru 17 down_at 33 last_clean_interval [17,32) [v2:127.0.0.1:6818/879957336,v1:127.0.0.1:6819/879957336] [v2:127.0.0.1:6820/879957336,v1:127.0.0.1:6821/879957336] exists,up 5a91b172-e692-4d4c-8a50-88f049a2dad1 2026-02-23T08:26:14.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:14.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:14.930 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:14.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-23T08:26:14.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:26:14.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:26:14.931 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:26:14.931 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:26:14.931 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:26:14.931 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:26:14.932 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:26:14.932 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:26:14.932 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:26:15.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:26:15.035 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:26:15.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:26:15.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:26:15.036 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:26:15.036 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:26:15.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:26:15.256 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:15.256 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:26:15.256 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:26:15.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:26:15.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:15.256 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:26:15.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-23T08:26:15.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-23T08:26:15.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786' 2026-02-23T08:26:15.336 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:15.337 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:26:15.416 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607560 2026-02-23T08:26:15.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607560 2026-02-23T08:26:15.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607560' 2026-02-23T08:26:15.417 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:15.417 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:26:15.498 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659139 2026-02-23T08:26:15.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659139 2026-02-23T08:26:15.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607560 2-176093659139' 2026-02-23T08:26:15.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:15.499 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:26:15.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182406 2026-02-23T08:26:15.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182406 2026-02-23T08:26:15.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-51539607560 2-176093659139 3-107374182406' 2026-02-23T08:26:15.577 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:15.578 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803786 2026-02-23T08:26:15.578 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:15.579 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:26:15.580 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-23T08:26:15.580 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:15.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803786 2026-02-23T08:26:15.581 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-23T08:26:15.582 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803786 2026-02-23T08:26:15.582 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:26:15.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803786 2026-02-23T08:26:15.800 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:26:16.801 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:26:16.802 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:26:17.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-23T08:26:17.018 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:17.019 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607560 2026-02-23T08:26:17.019 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:17.020 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:26:17.021 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607560 2026-02-23T08:26:17.021 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:17.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607560 2026-02-23T08:26:17.023 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607560' 2026-02-23T08:26:17.023 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 51539607560 2026-02-23T08:26:17.023 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:26:17.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607560 -lt 51539607560 2026-02-23T08:26:17.254 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:17.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-176093659139 2026-02-23T08:26:17.255 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:17.256 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:26:17.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-176093659139 2026-02-23T08:26:17.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:17.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659139 2026-02-23T08:26:17.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 176093659139' 2026-02-23T08:26:17.259 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 176093659139 2026-02-23T08:26:17.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:26:17.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-23T08:26:17.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:17.477 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182406 2026-02-23T08:26:17.477 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:17.478 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:26:17.479 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182406 2026-02-23T08:26:17.479 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:17.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182406 2026-02-23T08:26:17.481 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-23T08:26:17.481 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 107374182406 2026-02-23T08:26:17.482 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:26:17.701 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182407 -lt 107374182406 2026-02-23T08:26:17.702 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:26:17.702 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:17.702 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:17.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:26:17.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:26:17.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:26:17.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:26:17.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:26:17.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:26:17.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:26:17.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:26:18.217 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:26:18.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:26:18.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:18.218 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:18.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:26:18.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:26:18.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:26:18.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:26:18.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:26:18.509 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:26:18.765 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:26:18.774 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:26:19.047 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:26:19.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:26:19.057 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:26:19.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:26:19.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:26:19.058 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:26:19.058 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:26:19.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:26:19.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:26:19.058 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:26:19.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:26:19.371 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:26:19.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:26:19.372 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:26:19.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:26:19.374 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:26:19.375 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:26:19.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:19.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:26:19.376 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:26:19.377 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:19.378 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:26:19.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:26:19.379 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:26:19.394 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:26:19.394 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:19.394 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:19.394 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:26:19.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:26:19.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:26:19.396 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:26:19.397 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:26:19.399 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:26:19.399 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:26:19.400 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:26:19.401 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:26:19.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:26:19.402 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:26:19.402 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:26:19.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:19.403 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:26:19.404 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:26:19.404 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:19.405 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:26:19.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:26:19.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:26:19.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:26:19.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:19.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:19.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:26:19.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:26:19.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:26:19.410 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-23T08:26:19.411 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-23T08:26:19.411 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:19.412 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:19.412 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.74343 2026-02-23T08:26:19.413 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-23T08:26:19.414 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-23T08:26:19.415 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-23T08:26:19.436 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-23T08:26:19.436 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:19.436 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:19.436 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:19.436 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:19.437 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:19.437 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:19.437 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-23T08:26:19.467 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-23T08:26:19.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-23T08:26:19.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:26:19.470 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:26:19.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-23T08:26:19.471 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-23T08:26:19.471 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:26:19.471 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:26:19.471 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:26:19.471 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:19.471 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:19.472 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:19.472 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:26:19.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:26:19.472 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get fsid 2026-02-23T08:26:19.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-23T08:26:19.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-23T08:26:19.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-23T08:26:19.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-23T08:26:19.527 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-23T08:26:19.528 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-23T08:26:19.528 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:26:19.528 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:26:19.528 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:19.528 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:19.529 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:19.529 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:26:19.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-23T08:26:19.529 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.74343/ceph-mon.a.asok config get mon_host 2026-02-23T08:26:19.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-23T08:26:19.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-23T08:26:19.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-23T08:26:19.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-23T08:26:19.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-23T08:26:19.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-23T08:26:19.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-23T08:26:19.702 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-23T08:26:19.702 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:19.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:19.703 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:19.703 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:19.703 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:19.703 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:19.704 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:26:19.704 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-23T08:26:19.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-23T08:26:19.722 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-23T08:26:19.830 INFO:tasks.workunit.client.0.trial186.stderr:pool 'rbd' already exists 2026-02-23T08:26:19.838 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:26:20.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-23T08:26:20.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-23T08:26:20.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-23T08:26:20.840 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:20.840 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:20.840 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:20.840 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-mon.a.asok 2026-02-23T08:26:20.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-23T08:26:20.841 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-mon.a.asok log flush 2026-02-23T08:26:20.892 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:26:20.899 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-23T08:26:20.900 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:26:19.445+0000 7f2755d1bec0 0 load: jerasure load: lrc 2026-02-23T08:26:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-23T08:26:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-23T08:26:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-23T08:26:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-23T08:26:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-23T08:26:20.900 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-23T08:26:20.901 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-23T08:26:20.902 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-23T08:26:20.903 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:20.904 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:26:20.905 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-23T08:26:20.906 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:26:20.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e91d35a2-5c66-4375-9d12-8d34534d486d 2026-02-23T08:26:20.907 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e91d35a2-5c66-4375-9d12-8d34534d486d' 2026-02-23T08:26:20.907 INFO:tasks.workunit.client.0.trial186.stdout:add osd0 e91d35a2-5c66-4375-9d12-8d34534d486d 2026-02-23T08:26:20.907 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:26:20.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAsD5xpZ+BZNhAAV522fOi/mYHwzLYt6LnhvQ== 2026-02-23T08:26:20.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAsD5xpZ+BZNhAAV522fOi/mYHwzLYt6LnhvQ=="}' 2026-02-23T08:26:20.919 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e91d35a2-5c66-4375-9d12-8d34534d486d -i td/test-erasure-eio/0/new.json 2026-02-23T08:26:21.035 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:21.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-23T08:26:21.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAsD5xpZ+BZNhAAV522fOi/mYHwzLYt6LnhvQ== --osd-uuid e91d35a2-5c66-4375-9d12-8d34534d486d 2026-02-23T08:26:21.061 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.049+0000 7f04e5bada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:21.062 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.049+0000 7f04e5bada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:21.063 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.053+0000 7f04e5bada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:21.063 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.053+0000 7f04e5bada40 -1 bdev(0x55cd1b148400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-23T08:26:21.064 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.053+0000 7f04e5bada40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-23T08:26:21.729 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-23T08:26:21.730 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:26:21.731 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-23T08:26:21.731 INFO:tasks.workunit.client.0.trial186.stdout:adding osd0 key to auth repository 2026-02-23T08:26:21.731 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:26:21.846 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-23T08:26:21.847 INFO:tasks.workunit.client.0.trial186.stdout:start osd.0 2026-02-23T08:26:21.847 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:26:21.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:26:21.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:26:21.848 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:26:21.868 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.857+0000 7ff62abc0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:21.870 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.857+0000 7ff62abc0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:21.871 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:21.861+0000 7ff62abc0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:21.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-23T08:26:21.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:26:21.981 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-23T08:26:21.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:26:21.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:26:21.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:21.982 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:26:21.983 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:21.983 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:21.984 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:26:22.089 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:22.285 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:22.273+0000 7ff62abc0a40 -1 Falling back to public interface 2026-02-23T08:26:22.829 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:22.817+0000 7ff62abc0a40 -1 osd.0 0 log_to_monitors true 2026-02-23T08:26:23.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:23.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:23.091 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:23.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:23.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:23.091 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:26:23.306 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:24.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:24.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:24.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:26:24.307 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:26:24.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:24.307 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-23T08:26:24.517 INFO:tasks.workunit.client.0.trial186.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/938054362,v1:127.0.0.1:6803/938054362] [v2:127.0.0.1:6804/938054362,v1:127.0.0.1:6805/938054362] exists,up e91d35a2-5c66-4375-9d12-8d34534d486d 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:26:24.518 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:26:24.519 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:26:24.520 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-23T08:26:24.521 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:26:24.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e5106285-2bbb-4fc8-b70c-d987ee3656ca 2026-02-23T08:26:24.522 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e5106285-2bbb-4fc8-b70c-d987ee3656ca' 2026-02-23T08:26:24.522 INFO:tasks.workunit.client.0.trial186.stdout:add osd1 e5106285-2bbb-4fc8-b70c-d987ee3656ca 2026-02-23T08:26:24.523 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:26:24.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAwD5xp6CtsHxAA3QrdEQ7oBcig9mIspuhDtg== 2026-02-23T08:26:24.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAwD5xp6CtsHxAA3QrdEQ7oBcig9mIspuhDtg=="}' 2026-02-23T08:26:24.535 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e5106285-2bbb-4fc8-b70c-d987ee3656ca -i td/test-erasure-eio/1/new.json 2026-02-23T08:26:24.753 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:24.762 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-23T08:26:24.763 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAwD5xp6CtsHxAA3QrdEQ7oBcig9mIspuhDtg== --osd-uuid e5106285-2bbb-4fc8-b70c-d987ee3656ca 2026-02-23T08:26:24.780 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:24.769+0000 7f20b4492a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:24.781 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:24.769+0000 7f20b4492a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:24.782 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:24.769+0000 7f20b4492a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:24.783 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:24.769+0000 7f20b4492a40 -1 bdev(0x556b9df20400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-23T08:26:24.783 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:24.773+0000 7f20b4492a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-23T08:26:25.446 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-23T08:26:25.446 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:26:25.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-23T08:26:25.448 INFO:tasks.workunit.client.0.trial186.stdout:adding osd1 key to auth repository 2026-02-23T08:26:25.448 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:26:25.737 INFO:tasks.workunit.client.0.trial186.stdout:start osd.1 2026-02-23T08:26:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-23T08:26:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:26:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:26:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:26:25.738 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:26:25.762 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:25.748+0000 7f18163d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:25.763 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:25.752+0000 7f18163d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:25.764 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:25.752+0000 7f18163d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-23T08:26:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:26:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-23T08:26:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:26:25.958 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:26:25.959 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:25.959 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:25.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:26:25.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:25.960 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:26:26.101 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:26.088+0000 7f18163d1a40 -1 Falling back to public interface 2026-02-23T08:26:26.168 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:26.701 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:26.688+0000 7f18163d1a40 -1 osd.1 0 log_to_monitors true 2026-02-23T08:26:27.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:27.169 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:27.169 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:27.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:27.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:27.170 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-23T08:26:27.388 INFO:tasks.workunit.client.0.trial186.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2242793293,v1:127.0.0.1:6811/2242793293] [v2:127.0.0.1:6812/2242793293,v1:127.0.0.1:6813/2242793293] exists,up e5106285-2bbb-4fc8-b70c-d987ee3656ca 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:26:27.389 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:26:27.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:26:27.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:27.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:27.390 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:27.390 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:27.390 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:27.390 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:26:27.391 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:26:27.392 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:26:27.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2bd4505b-bd5d-4d69-a71d-a198bde9d3e3 2026-02-23T08:26:27.393 INFO:tasks.workunit.client.0.trial186.stdout:add osd2 2bd4505b-bd5d-4d69-a71d-a198bde9d3e3 2026-02-23T08:26:27.393 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2bd4505b-bd5d-4d69-a71d-a198bde9d3e3' 2026-02-23T08:26:27.394 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:26:27.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAzD5xp0MK8FxAAlRg5ltEoBNZxUajeYfbpDw== 2026-02-23T08:26:27.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAzD5xp0MK8FxAAlRg5ltEoBNZxUajeYfbpDw=="}' 2026-02-23T08:26:27.406 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2bd4505b-bd5d-4d69-a71d-a198bde9d3e3 -i td/test-erasure-eio/2/new.json 2026-02-23T08:26:27.615 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:26:27.624 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-23T08:26:27.625 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAzD5xp0MK8FxAAlRg5ltEoBNZxUajeYfbpDw== --osd-uuid 2bd4505b-bd5d-4d69-a71d-a198bde9d3e3 2026-02-23T08:26:27.642 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:27.628+0000 7f508f503a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:27.643 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:27.628+0000 7f508f503a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:27.644 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:27.632+0000 7f508f503a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:27.645 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:27.632+0000 7f508f503a40 -1 bdev(0x56419fcbe400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-23T08:26:27.645 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:27.632+0000 7f508f503a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-23T08:26:28.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-23T08:26:28.258 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:26:28.259 INFO:tasks.workunit.client.0.trial186.stdout:adding osd2 key to auth repository 2026-02-23T08:26:28.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-23T08:26:28.260 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:26:28.546 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:26:28.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-23T08:26:28.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:26:28.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:26:28.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:26:28.548 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:26:28.570 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:28.556+0000 7f51ee991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:28.571 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:28.560+0000 7f51ee991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:28.573 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:28.560+0000 7f51ee991a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:28.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-23T08:26:28.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:26:28.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:26:28.766 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:26:28.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:26:28.767 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:28.767 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:28.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:26:28.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:28.768 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:28.969 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:28.956+0000 7f51ee991a40 -1 Falling back to public interface 2026-02-23T08:26:28.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:29.629 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:29.616+0000 7f51ee991a40 -1 osd.2 0 log_to_monitors true 2026-02-23T08:26:29.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:29.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:29.992 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:29.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:29.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:29.992 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:30.208 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:31.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:31.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:31.209 INFO:tasks.workunit.client.0.trial186.stdout:2 2026-02-23T08:26:31.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-23T08:26:31.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:31.209 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:31.423 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2942484229,v1:127.0.0.1:6819/2942484229] [v2:127.0.0.1:6820/2942484229,v1:127.0.0.1:6821/2942484229] exists,up 2bd4505b-bd5d-4d69-a71d-a198bde9d3e3 2026-02-23T08:26:31.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:31.424 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:31.425 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:26:31.426 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:26:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:26:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:26:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:26:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-23T08:26:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-23T08:26:31.427 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-23T08:26:31.427 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-23T08:26:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=783004bb-9f75-44ea-a04a-15a15b3fc1d0 2026-02-23T08:26:31.428 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 783004bb-9f75-44ea-a04a-15a15b3fc1d0' 2026-02-23T08:26:31.428 INFO:tasks.workunit.client.0.trial186.stdout:add osd3 783004bb-9f75-44ea-a04a-15a15b3fc1d0 2026-02-23T08:26:31.428 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-23T08:26:31.440 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA3D5xpHfLKGRAAjHohreWDuyqMOk7IxOlfdw== 2026-02-23T08:26:31.441 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA3D5xpHfLKGRAAjHohreWDuyqMOk7IxOlfdw=="}' 2026-02-23T08:26:31.441 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 783004bb-9f75-44ea-a04a-15a15b3fc1d0 -i td/test-erasure-eio/3/new.json 2026-02-23T08:26:31.667 INFO:tasks.workunit.client.0.trial186.stdout:3 2026-02-23T08:26:31.677 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-23T08:26:31.678 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA3D5xpHfLKGRAAjHohreWDuyqMOk7IxOlfdw== --osd-uuid 783004bb-9f75-44ea-a04a-15a15b3fc1d0 2026-02-23T08:26:31.694 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:31.680+0000 7f3a80deda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:31.695 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:31.684+0000 7f3a80deda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:31.696 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:31.684+0000 7f3a80deda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:31.697 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:31.684+0000 7f3a80deda40 -1 bdev(0x555c7e0aa400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-23T08:26:31.697 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:31.684+0000 7f3a80deda40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-23T08:26:32.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-23T08:26:32.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-23T08:26:32.327 INFO:tasks.workunit.client.0.trial186.stdout:adding osd3 key to auth repository 2026-02-23T08:26:32.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-23T08:26:32.327 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-23T08:26:32.616 INFO:tasks.workunit.client.0.trial186.stdout:start osd.3 2026-02-23T08:26:32.617 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-23T08:26:32.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:26:32.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-23T08:26:32.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-23T08:26:32.618 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-23T08:26:32.639 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:32.624+0000 7f7cd2ed5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:32.641 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:32.628+0000 7f7cd2ed5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:32.642 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:32.628+0000 7f7cd2ed5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:32.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-23T08:26:32.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:26:32.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-23T08:26:32.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:26:32.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:26:32.834 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:32.834 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:32.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:26:32.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:32.835 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:26:33.013 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:33.000+0000 7f7cd2ed5a40 -1 Falling back to public interface 2026-02-23T08:26:33.043 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:33.613 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:33.600+0000 7f7cd2ed5a40 -1 osd.3 0 log_to_monitors true 2026-02-23T08:26:34.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:34.044 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:34.045 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:34.045 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:34.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:34.046 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-23T08:26:34.256 INFO:tasks.workunit.client.0.trial186.stdout:osd.3 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/241242134,v1:127.0.0.1:6827/241242134] [v2:127.0.0.1:6828/241242134,v1:127.0.0.1:6829/241242134] exists,up 783004bb-9f75-44ea-a04a-15a15b3fc1d0 2026-02-23T08:26:34.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:34.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:34.257 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:34.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-23T08:26:34.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-23T08:26:34.257 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-23T08:26:34.258 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:34.259 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:34.259 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:34.259 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.0.asok 2026-02-23T08:26:34.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-23T08:26:34.259 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.0.asok log flush 2026-02-23T08:26:34.322 INFO:tasks.workunit.client.0.trial186.stdout:{} 2026-02-23T08:26:34.328 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-23T08:26:34.330 INFO:tasks.workunit.client.0.trial186.stdout:2026-02-23T08:26:22.277+0000 7ff62abc0a40 0 load: jerasure load: lrc 2026-02-23T08:26:34.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-23T08:26:34.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-23T08:26:34.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:26:34.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-23T08:26:34.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-23T08:26:34.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-23T08:26:34.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-23T08:26:34.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-23T08:26:34.332 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-23T08:26:34.583 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-23T08:26:34.584 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-23T08:26:34.980 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' already exists 2026-02-23T08:26:34.991 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-23T08:26:35.993 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-23T08:26:35.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:26:35.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:26:35.994 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:26:35.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:26:35.995 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:26:35.995 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:26:35.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:26:35.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:26:35.995 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:26:36.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:26:36.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:26:36.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:26:36.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:26:36.105 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:26:36.105 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:26:36.325 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:26:36.326 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:36.326 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:26:36.326 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:26:36.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:26:36.326 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:36.326 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:26:36.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-23T08:26:36.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-23T08:26:36.407 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-23T08:26:36.408 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:36.408 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:26:36.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-23T08:26:36.492 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-23T08:26:36.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-23T08:26:36.493 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:36.493 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:26:36.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-23T08:26:36.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-23T08:26:36.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411332' 2026-02-23T08:26:36.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:36.568 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:26:36.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-23T08:26:36.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-23T08:26:36.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411332 3-115964116995' 2026-02-23T08:26:36.649 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:36.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-23T08:26:36.649 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:36.651 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:26:36.651 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-23T08:26:36.651 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:36.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-23T08:26:36.653 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-23T08:26:36.653 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803781 2026-02-23T08:26:36.654 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:26:36.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-23T08:26:36.873 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:26:37.875 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:26:37.875 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:26:38.093 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-23T08:26:38.093 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:38.094 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574852 2026-02-23T08:26:38.094 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:38.095 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:26:38.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-23T08:26:38.096 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:38.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574852 2026-02-23T08:26:38.098 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-23T08:26:38.098 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574852 2026-02-23T08:26:38.099 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:26:38.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574852 2026-02-23T08:26:38.319 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:38.320 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-23T08:26:38.320 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:38.321 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:26:38.322 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-23T08:26:38.322 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:38.323 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-23T08:26:38.324 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-23T08:26:38.324 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 77309411332 2026-02-23T08:26:38.324 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:26:38.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-23T08:26:38.545 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:38.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116995 2026-02-23T08:26:38.546 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:38.547 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:26:38.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-23T08:26:38.548 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:38.549 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116995 2026-02-23T08:26:38.550 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 115964116995 2026-02-23T08:26:38.550 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-23T08:26:38.550 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:26:38.779 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-23T08:26:38.780 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:26:38.780 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:38.781 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:26:39.071 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:26:39.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:26:39.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:26:39.290 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:39.291 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:39.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:26:39.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:26:39.586 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-74343-1 2026-02-23T08:26:39.587 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-74343-1 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-74343-1 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-23T08:26:39.588 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-74343-1 td/test-erasure-eio/ORIGINAL 2026-02-23T08:26:39.607 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-74343-1 td/test-erasure-eio 1 2026-02-23T08:26:39.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-23T08:26:39.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-23T08:26:39.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-23T08:26:39.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-23T08:26:39.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-23T08:26:39.608 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-74343-1 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-74343-1 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:26:39.609 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-1 2026-02-23T08:26:39.610 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-1 2026-02-23T08:26:39.610 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:26:39.823 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:26:39.823 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:39.824 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:26:39.824 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:26:39.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-23T08:26:39.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-23T08:26:39.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-23T08:26:39.824 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-23T08:26:39.825 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-23T08:26:39.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-23T08:26:39.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-23T08:26:39.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-23T08:26:39.825 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-23T08:26:39.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-23T08:26:39.826 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-23T08:26:39.826 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-23T08:26:39.826 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-23T08:26:39.827 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:26:39.827 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:26:39.827 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:39.828 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:39.828 INFO:tasks.workunit.client.0.trial186.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:39.828 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:26:39.828 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.74343/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-23T08:26:39.890 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-23T08:26:39.891 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-23T08:26:39.892 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-23T08:26:39.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-23T08:26:39.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-23T08:26:39.892 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-23T08:26:39.892 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-23T08:26:39.892 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:39.892 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:39.893 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.74343/ceph-osd.1.asok 2026-02-23T08:26:39.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-23T08:26:39.893 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.74343/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-74343-1 1 2026-02-23T08:26:39.953 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-74343-1 2026-02-23T08:26:39.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-23T08:26:39.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-23T08:26:39.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-74343-1 2026-02-23T08:26:39.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-23T08:26:39.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-23T08:26:39.954 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-74343-1 td/test-erasure-eio/COPY 2026-02-23T08:26:39.972 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-23T08:26:39.973 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-23T08:26:39.974 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-23T08:26:39.975 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-74343-1 2026-02-23T08:26:39.975 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:26:39.975 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-1 2026-02-23T08:26:39.976 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-1 2026-02-23T08:26:39.976 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:26:40.196 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:26:40.196 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:40.196 INFO:tasks.workunit.client.0.trial186.stderr:2' 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:26:40.197 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:26:40.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:26:40.505 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-23T08:26:40.773 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already out. 2026-02-23T08:26:40.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-74343-1 2026-02-23T08:26:40.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-23T08:26:40.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-23T08:26:40.784 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-74343-1 2026-02-23T08:26:40.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-74343-1 2026-02-23T08:26:40.785 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-23T08:26:41.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-23T08:26:41.006 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:41.006 INFO:tasks.workunit.client.0.trial186.stderr:0' 2026-02-23T08:26:41.006 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-23T08:26:41.007 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-23T08:26:41.275 INFO:tasks.workunit.client.0.trial186.stderr:osd.2 is already in. 2026-02-23T08:26:41.287 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-23T08:26:41.288 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-23T08:26:41.289 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-23T08:26:41.290 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-23T08:26:41.291 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-23T08:26:41.291 INFO:tasks.workunit.client.0.trial186.stdout:start osd.2 2026-02-23T08:26:41.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=6e06db35-9e80-475b-935b-4015473c69fb --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.74343/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-23T08:26:41.292 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-23T08:26:41.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-23T08:26:41.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-23T08:26:41.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-23T08:26:41.292 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-23T08:26:41.303 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:41.288+0000 7effef73ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:41.304 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:41.292+0000 7effef73ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:41.305 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:41.292+0000 7effef73ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stdout:0 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:41.499 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:41.709 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-23T08:26:41.781 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:41.768+0000 7effef73ca40 -1 Falling back to public interface 2026-02-23T08:26:42.363 INFO:tasks.workunit.client.0.trial186.stderr:2026-02-23T08:26:42.348+0000 7effef73ca40 -1 osd.2 35 log_to_monitors true 2026-02-23T08:26:42.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-23T08:26:42.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-23T08:26:42.711 INFO:tasks.workunit.client.0.trial186.stdout:1 2026-02-23T08:26:42.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-23T08:26:42.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-23T08:26:42.711 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-23T08:26:42.919 INFO:tasks.workunit.client.0.trial186.stdout:osd.2 up in weight 1 up_from 44 up_thru 20 down_at 36 last_clean_interval [18,35) [v2:127.0.0.1:6818/1029712258,v1:127.0.0.1:6819/1029712258] [v2:127.0.0.1:6820/1029712258,v1:127.0.0.1:6821/1029712258] exists,up 2bd4505b-bd5d-4d69-a71d-a198bde9d3e3 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-23T08:26:42.921 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-23T08:26:42.922 INFO:tasks.workunit.client.0.trial186.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-23T08:26:42.922 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-23T08:26:42.922 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-23T08:26:42.922 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-23T08:26:43.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-23T08:26:43.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-23T08:26:43.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-23T08:26:43.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-23T08:26:43.031 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-23T08:26:43.032 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-23T08:26:43.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-23T08:26:43.252 INFO:tasks.workunit.client.0.trial186.stderr:1 2026-02-23T08:26:43.252 INFO:tasks.workunit.client.0.trial186.stderr:2 2026-02-23T08:26:43.252 INFO:tasks.workunit.client.0.trial186.stderr:3' 2026-02-23T08:26:43.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-23T08:26:43.252 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:43.252 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-23T08:26:43.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-23T08:26:43.330 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-23T08:26:43.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-23T08:26:43.331 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:43.331 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-23T08:26:43.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-23T08:26:43.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-23T08:26:43.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574856' 2026-02-23T08:26:43.409 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:43.409 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-23T08:26:43.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561027 2026-02-23T08:26:43.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561027 2026-02-23T08:26:43.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-188978561027' 2026-02-23T08:26:43.489 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-23T08:26:43.490 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-23T08:26:43.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116998 2026-02-23T08:26:43.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116998 2026-02-23T08:26:43.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-188978561027 3-115964116998' 2026-02-23T08:26:43.568 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:43.569 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-23T08:26:43.569 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:43.570 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-23T08:26:43.570 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-23T08:26:43.571 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:43.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-23T08:26:43.572 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-23T08:26:43.572 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.0 seq 25769803784 2026-02-23T08:26:43.573 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:26:43.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-23T08:26:43.791 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-23T08:26:44.792 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-23T08:26:44.793 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-23T08:26:45.003 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-23T08:26:45.004 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:45.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574856 2026-02-23T08:26:45.004 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:45.005 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-23T08:26:45.006 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574856 2026-02-23T08:26:45.007 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:45.008 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-23T08:26:45.008 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.1 seq 55834574856 2026-02-23T08:26:45.009 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574856' 2026-02-23T08:26:45.009 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-23T08:26:45.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-23T08:26:45.231 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:45.231 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-188978561027 2026-02-23T08:26:45.231 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:45.233 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-23T08:26:45.234 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-188978561027 2026-02-23T08:26:45.234 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:45.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561027 2026-02-23T08:26:45.235 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.2 seq 188978561027 2026-02-23T08:26:45.235 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 188978561027' 2026-02-23T08:26:45.236 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-23T08:26:45.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561027 -lt 188978561027 2026-02-23T08:26:45.447 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-23T08:26:45.447 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116998 2026-02-23T08:26:45.447 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-23T08:26:45.449 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-23T08:26:45.449 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116998 2026-02-23T08:26:45.450 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-23T08:26:45.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116998 2026-02-23T08:26:45.451 INFO:tasks.workunit.client.0.trial186.stdout:waiting osd.3 seq 115964116998 2026-02-23T08:26:45.451 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-23T08:26:45.452 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-23T08:26:45.667 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116998 -lt 115964116998 2026-02-23T08:26:45.667 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-23T08:26:45.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:45.668 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:45.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-23T08:26:45.961 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-23T08:26:45.961 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-23T08:26:45.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-23T08:26:45.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-23T08:26:45.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-23T08:26:45.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-23T08:26:45.962 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-23T08:26:46.173 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-23T08:26:46.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-23T08:26:46.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-23T08:26:46.174 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-23T08:26:46.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-23T08:26:46.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-23T08:26:46.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-23T08:26:46.475 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-23T08:26:46.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-23T08:26:46.476 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-23T08:26:46.732 INFO:tasks.workunit.client.0.trial186.stderr:pool 'pool-jerasure' does not exist 2026-02-23T08:26:46.741 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-23T08:26:47.012 INFO:tasks.workunit.client.0.trial186.stderr:erasure-code-profile myprofile does not exist 2026-02-23T08:26:47.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-23T08:26:47.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:26:47.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-23T08:26:47.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:26:47.022 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:26:47.022 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:26:47.022 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:26:47.023 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:26:47.023 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:26:47.333 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:26:47.334 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:26:47.335 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:26:47.335 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:26:47.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:26:47.337 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:26:47.337 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:26:47.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:47.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:26:47.339 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:26:47.339 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:47.340 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:26:47.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-23T08:26:47.341 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:26:47.355 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:26:47.355 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:47.355 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:47.355 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-eio 0 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-23T08:26:47.357 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-23T08:26:47.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-23T08:26:47.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-23T08:26:47.358 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-23T08:26:47.360 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-23T08:26:47.361 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-23T08:26:47.362 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-23T08:26:47.362 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-23T08:26:47.363 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-23T08:26:47.364 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-23T08:26:47.364 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-23T08:26:47.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:47.365 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-23T08:26:47.366 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-23T08:26:47.367 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-23T08:26:47.367 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:26:47.368 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-23T08:26:47.369 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-23T08:26:47.370 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-23T08:26:47.370 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-23T08:26:47.370 INFO:tasks.workunit.client.0.trial186.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.74343 2026-02-23T08:26:47.370 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.74343 2026-02-23T08:26:47.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-23T08:26:47.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-23T08:26:47.371 INFO:tasks.workunit.client.0.trial186.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-23T08:26:47.372 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-23T08:26:47.373 DEBUG:teuthology.orchestra.run.trial186:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-23T08:26:47.423 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-23T08:26:47.424 DEBUG:teuthology.orchestra.run.trial186:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-23T08:26:47.785 DEBUG:teuthology.parallel:result is None 2026-02-23T08:26:47.785 DEBUG:teuthology.orchestra.run.trial186:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-23T08:26:47.793 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-23T08:26:47.793 DEBUG:teuthology.orchestra.run.trial186:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-23T08:26:47.839 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-23T08:26:47.839 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-23T08:26:47.849 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-23T08:26:47.850 DEBUG:teuthology.orchestra.run.trial186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-23T08:26:47.987 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-23T08:26:47.999 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-23T08:26:47.999 DEBUG:teuthology.orchestra.run.trial186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-23T08:26:48.046 INFO:teuthology.orchestra.run.trial186.stdout: remote refid st t when poll reach delay offset jitter 2026-02-23T08:26:48.046 INFO:teuthology.orchestra.run.trial186.stdout:============================================================================== 2026-02-23T08:26:48.046 INFO:teuthology.orchestra.run.trial186.stdout:*ool-44c5de70.dy .PPS. 1 u 3 64 357 5.055 -0.576 5.449 2026-02-23T08:26:48.047 INFO:teuthology.orchestra.run.trial186.stdout:+as393746.mci.tr 42.20.202.230 2 u 10 64 377 29.840 +2.993 2.984 2026-02-23T08:26:48.047 INFO:teuthology.orchestra.run.trial186.stdout:-172-233-157-223 80.192.165.246 2 u 8 64 377 66.621 +12.564 15.993 2026-02-23T08:26:48.047 INFO:teuthology.orchestra.run.trial186.stdout:+sensei.ruselabs 38.6.228.170 2 u 4 64 377 68.463 +4.947 2.082 2026-02-23T08:26:48.047 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-23T08:26:48.059 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-23T08:26:48.059 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-23T08:26:48.072 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-23T08:26:48.085 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-23T08:26:48.098 INFO:teuthology.task.internal:Duration was 1276.429774 seconds 2026-02-23T08:26:48.098 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-23T08:26:48.113 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-23T08:26:48.114 DEBUG:teuthology.orchestra.run.trial186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-23T08:26:48.139 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-23T08:26:48.139 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial186.front.sepia.ceph.com 2026-02-23T08:26:48.139 DEBUG:teuthology.orchestra.run.trial186:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-23T08:26:48.185 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-23T08:26:48.186 DEBUG:teuthology.orchestra.run.trial186:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-23T08:26:48.450 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-23T08:26:48.451 DEBUG:teuthology.orchestra.run.trial186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-23T08:26:48.460 INFO:teuthology.orchestra.run.trial186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-23T08:26:48.460 INFO:teuthology.orchestra.run.trial186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-23T08:26:48.460 INFO:teuthology.orchestra.run.trial186.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-23T08:26:48.461 INFO:teuthology.orchestra.run.trial186.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-23T08:26:48.461 INFO:teuthology.orchestra.run.trial186.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-23T08:26:48.531 INFO:teuthology.orchestra.run.trial186.stderr: 81.3% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-23T08:26:48.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-23T08:26:48.544 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-23T08:26:48.544 DEBUG:teuthology.orchestra.run.trial186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-23T08:26:48.582 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-23T08:26:48.599 DEBUG:teuthology.orchestra.run.trial186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:26:48.631 INFO:teuthology.orchestra.run.trial186.stdout:kernel.core_pattern = core 2026-02-23T08:26:48.640 DEBUG:teuthology.orchestra.run.trial186:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-23T08:26:48.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T08:26:48.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-23T08:26:48.702 INFO:teuthology.task.internal:Transferring archived files... 2026-02-23T08:26:48.705 DEBUG:teuthology.misc:Transferring archived files from trial186:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-23_07:12:59-rados-wip-bharath8-testing-2026-02-22-0716-distro-default-trial/65627/remote/trial186 2026-02-23T08:26:48.706 DEBUG:teuthology.orchestra.run.trial186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-23T08:26:48.781 INFO:teuthology.task.internal:Removing archive directory... 2026-02-23T08:26:48.781 DEBUG:teuthology.orchestra.run.trial186:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-23T08:26:48.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-23T08:26:48.816 INFO:teuthology.task.internal:Not uploading archives. 2026-02-23T08:26:48.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-23T08:26:48.830 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-23T08:26:48.830 DEBUG:teuthology.orchestra.run.trial186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-23T08:26:48.846 INFO:teuthology.orchestra.run.trial186.stdout: 1179687 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 23 08:26 /home/ubuntu/cephtest 2026-02-23T08:26:48.847 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-23T08:26:48.859 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-23T08:26:48.892 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} duration: 1276.4297738075256 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-23T08:26:48.892 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-23T08:26:48.990 INFO:teuthology.run:pass